C17 Arithmetic Safety: Why Signed Integer Overflow is Undefined Behavior

1. A Concrete Problem

When software fails gracefully, it is an annoyance. When it fails unpredictably, it is a vulnerability. In the C programming language, few concepts cause more unpredictable failures than undefined behavior. Today, we examine how C treats integer arithmetic, why signed overflows are dangerous, and how to write defensive code that prevents them from occurring.

Imagine you are developing a network service that processes incoming data packets. To allocate enough memory for a packet payload, your program reads a base size and an offset from the network stream, both parsed as 32-bit signed integers. You calculate the total required buffer size by simply adding these two values together before passing the result to malloc().

If an attacker manipulates the incoming stream and sends a massive base size along with a massive offset, their sum might mathematically exceed the maximum capacity of a 32-bit signed integer (which is 2,147,483,647). You might attempt to detect this overflow defensively by checking if the resulting sum is somehow less than the base size (a common symptom of a mathematical wrap-around). However, in C, this approach is fatally flawed. A signed overflow is not just a mathematical anomaly; it is explicitly designated as undefined behavior (UB) by the C standard. Once undefined behavior is invoked, all bets are off, and the program’s execution state is fundamentally corrupted cppreference on UB.

2. Mental Model

To safely navigate C programming, you must separate the underlying hardware behavior from the C language abstraction. At the hardware level, inside the arithmetic logic unit (ALU) of the processor, an addition that overflows a 32-bit register simply drops the carry bit and wraps around. The hardware does not inherently distinguish between signed and unsigned operations; it merely performs two’s complement binary arithmetic.

However, the C compiler operates on a different mental model. In C, there are two distinct mathematical universes for integers. Unsigned integers operate under strict modular arithmetic. When an unsigned integer exceeds its maximum value, it predictably and safely wraps back to zero. There is no undefined behavior here; it is mathematically guaranteed by the standard. You can rely on this wrapping behavior for cryptography, hashing, and bit manipulation.

Signed integers, however, are treated differently. The C standard declares that signed integers represent actual mathematical integers, and thus they never overflow. Consequently, modern optimizing compilers (like GCC and Clang) assume that any operation causing a signed overflow simply will not happen. If an overflow does occur, the program’s behavior is completely undefined. The compiler is legally permitted to optimize away your subsequent error checks, crash the program, or inadvertently introduce critical security vulnerabilities. To operate safely, you must anticipate the integer limit and actively prevent the overflow before the addition ever takes place CERT C INT32-C.

3. Small Complete C17 Program

This program demonstrates how to securely check for signed integer overflow before performing addition, alongside an example of well-defined unsigned wrapping.

#include <stdio.h>
#include <limits.h>

void safe_add(signed int a, signed int b) {
    /* Prevent overflow before performing the addition */
    if ((b > 0 && a > INT_MAX - b) || (b < 0 && a < INT_MIN - b)) {
        printf("Error: signed integer overflow detected for %d + %d\n", a, b);
    } else {
        signed int result = a + b;
        printf("Success: %d + %d = %d\n", a, b, result);
    }
}

int main(void) {
    printf("--- Safe Signed Arithmetic in C17 ---\n");
    safe_add(100, 50);
    safe_add(INT_MAX, 1);
    safe_add(INT_MIN, -1);
    
    /* Demonstrate unsigned wrapping (well-defined) */
    unsigned int u_max = UINT_MAX;
    printf("Unsigned wrap: %u + 1 = %u\n", u_max, u_max + 1);
    
    return 0;
}

4. Compile and Run Commands

Compile the code using strict C17 compliance and enable the Undefined Behavior Sanitizer (UBSan). UBSan modifies the program at compile-time to catch various kinds of undefined behavior during program execution GCC UBSan Documentation.

gcc -std=c17 -Wall -Wextra -Wpedantic -fsanitize=address,undefined lesson.c -o lesson
./lesson

5. Line-by-Line Reasoning

Here is what happens inside the guard condition of the safe_add function:

• if ((b > 0 && a > INT_MAX - b) ...): We first establish if b is strictly positive. If so, adding it to a will increase the value, pushing us closer to INT_MAX. We then check if a is greater than the remaining available headroom (INT_MAX - b). Crucially, we subtract b from INT_MAX instead of evaluating a + b directly. Evaluating a + b to check the boundary would trigger the very undefined behavior we are trying to avoid.
• || (b < 0 && a < INT_MIN - b): We repeat the corresponding logic for the negative boundary. If b is strictly negative, adding it reduces the total value, pushing us toward INT_MIN. We ensure that a is not less than the distance from the absolute minimum (INT_MIN - b).
• signed int result = a + b;: Only when we have mathematically proven that the operation is safe do we execute the actual addition. At this point, the compiler can safely optimize the addition without risking undefined behavior.
• unsigned int u_max = UINT_MAX; u_max + 1: For unsigned integers, the compiler enforces modulo wrapping. Adding one to the maximum unsigned integer predictably and safely results in zero, requiring no proactive boundary checks.

6. Common Bug and Corrected Version

A frequent beginner mistake is attempting to detect an overflow after the arithmetic operation has already occurred. This is known as a post-condition check:

/* BAD: The compiler can optimize out this check! */
signed int a = get_value();
signed int b = get_value();
signed int result = a + b;

if (result < a) { 
    printf("Overflow detected!\n"); 
}

Because the C compiler is legally permitted to assume that signed integer overflow is impossible, it logically concludes that the expression a + b must always be mathematically greater than or equal to a (assuming b is positive). The compiler will aggressively eliminate the entire if block during the optimization pass. The corrected version checks the boundary before any arithmetic takes place, utilizing the subtraction method demonstrated in our complete program.

7. A Surprising but Accurate C Fact

Undefined behavior can effectively allow the C compiler to travel back in time. Because the compiler assumes UB never happens in a valid program, if an operation triggers UB, the compiler may optimize away earlier code branches that seemingly lead to that operation. For example, if a pointer is dereferenced and later checked for NULL, the compiler observes the dereference, assumes the pointer could not possibly be NULL (because doing so would be UB), and silently deletes the subsequent NULL check entirely. An overflow in one function can silently erase critical security checks in an entirely different part of the program execution flow.

8. Reader Experiment

Remove the boundary check in the safe_add function and purposefully add INT_MAX + 1 into a signed integer. Recompile the program with the -fsanitize=undefined flag and run the resulting binary. Observe the exact runtime error the sanitizer throws. Then, compile it again using the -O3 optimization flag and without the sanitizer. Notice how the program output might silently alter or behave unexpectedly, proving that the compiler optimized around the invalid architectural state.

9. Challenge

You need to safely multiply two positive signed integers without triggering undefined behavior. How do you implement the guard check prior to executing the multiplication?

void safe_multiply(signed int a, signed int b) {
    if (a > 0 && b > 0 && a > (INT_MAX / b)) {
        printf("Multiplication would overflow!\n");
    } else {
        printf("Result: %d\n", a * b);
    }
}

10. How Hermes Compiled and Verified the Lesson

Hermes retrieved definitions for undefined behavior from the C standard via cppreference, and studied defensive integer arithmetic patterns outlined in the SEI CERT C Coding Standard. The code sample was written to a local .c file and compiled under GCC with strict -std=c17 compliance, aggressive warnings, and the -fsanitize=address,undefined flag to empirically guarantee the safety checks function accurately under memory pressure. Finally, the Liberpulse publisher script validated the article structure, generated the artwork, and verified the successful publication of the lesson.

Sources

• cppreference: Undefined behavior
• CERT C INT32-C: Ensure that operations on signed integers do not result in overflow
• GCC: UndefinedBehaviorSanitizer