openlang

Why did the compiler delete my signed overflow check?

if (x + 1 < x) on a signed int is not an overflow guard. Signed overflow is undefined behavior in C, so the compiler may assume x + 1 never wraps, conclude the condition is false, and drop the branch. Mainstream compilers do exactly that once optimization is on.

This is a security problem because the vanishing check is usually the one in front of a length passed to malloc or memcpy. The source reads safe in review. The binary has no check in it.

Check before the operation. if (x > INT_MAX - 1) for the simple case. For general addition, __builtin_add_overflow in GCC and Clang, or ckd_add from <stdckdint.h> if your toolchain has C23. -fwrapv makes signed overflow wrap on compilers that support it, which is a fine mitigation and not portable C. And unsigned arithmetic such as size_t wraps by definition, which only means you get a well-defined wrong length, so compare against the limit explicitly there too.

The rule is one sentence in section 6.5 of the standard. It explains a lot of bug reports that blame the optimizer.

The conversation continues 1

Follow the reasoning
  1. Permalink

    Swift went the other way and I like it. + on Int traps on overflow, so the bad length crashes instead of sailing through. If you want wrapping you have to say so with &+, and addingReportingOverflow hands back the result plus a flag when you'd rather check than crash. A crash isn't free either. It's still much easier to find than a check that silently vanished. (Unless someone builds with -Ounchecked, which removes those checks.)

Other sides of the same question

  1. Should a Rego allow rule default to false?

    {"approved": false} and {} get the same answer from allow if input.approved == true. The answer is undefined, not false. The…

    Replies 2regosql