This C function should always return false, but it doesn’t

I stumbled over an interesting question in a forum a long time ago and I want to know the answer.

Consider the following C function:

f1.c

#include <stdbool.h>

bool f1()
{
	int var1 = 1000;
	int var2 = 2000;
	int var3 = var1 + var2;
	return (var3 == 0) ? true : false;
}

This should always return false since var3 == 3000. The main function looks like this:

main.c

#include <stdio.h>
#include <stdbool.h>

int main()
{
	printf( f1() == true ? "true\n" : "false\n");
	if( f1() )
	{
		printf("executed\n");
	}
	return 0;
}

Since f1() should always return false, one would expect the program to print only one false to the screen. But after compiling and running it, executed is also displayed:

$ gcc main.c f1.c -o test
$ ./test
false
executed

Why is that? Does this code have some sort of undefined behavior?

Note: I compiled it with gcc (Ubuntu 4.9.2-10ubuntu13) 4.9.2.

As noted in other answers, the problem is that you use gcc with no compiler options set. If you do this, it defaults to what is called "gnu90", which is a non-standard implementation of the old, withdrawn C90 standard from 1990.

In the old C90 standard there was a major flaw in the C language: if you didn't declare a prototype before using a function, it would default to int func () (where ( ) means "accept any parameter"). This changes the calling convention of the function func, but it doesn't change the actual function definition. Since the size of bool and int are different, your code invokes undefined behavior when the function is called.

This dangerous nonsense behavior was fixed in the year 1999, with the release of the C99 standard. Implicit function declarations were banned.

Unfortunately, GCC up to version 5.x.x still uses the old C standard by default. There is probably no reason why you should want to compile your code as anything but standard C. So you have to explicitly tell GCC that it should compile your code as modern C code, instead of some 25+ years old, non-standard GNU crap.

Fix the problem by always compiling your program as:

gcc -std=c11 -pedantic-errors -Wall -Wextra

• -std=c11 tells it to make a half-hearted attempt to compile according the (current) C standard (informally known as C11).
• -pedantic-errors tells it to whole-heartedly do the above, and give compiler errors when you write incorrect code which violates the C standard.
• -Wall means give me some extra warnings that might be good to have.
• -Wextra means give me some other extra warnings that might be good to have.

You don't have a prototype declared for f1() in main.c, so it is implicitly defined as int f1(), meaning it is a function that takes an unknown number of arguments and returns an int.

If int and bool are of different sizes, this will result in undefined behavior. For example, on my machine, int is 4 bytes and bool is one byte. Since the function is defined to return bool, it puts one byte on the stack when it returns. However, since it's implicitly declared to return int from main.c, the calling function will try to read 4 bytes from the stack.

The default compilers options in gcc won't tell you that it's doing this. But if you compile with -Wall -Wextra, you'll get this:

main.c: In function ‘main’:
main.c:6: warning: implicit declaration of function ‘f1’

To fix this, add a declaration for f1 in main.c, before main:

bool f1(void);

Note that the argument list is explicitly set to void, which tells the compiler the function takes no arguments, as opposed to an empty parameter list which means an unknown number of arguments. The definition f1 in f1.c should also be changed to reflect this.

I think it's interesting to see where the size-mismatch mentioned in Lundin's excellent answer actually happens.

If you compile with --save-temps, you will get assembly files that you can look at. Here's the part where f1() does the == 0 comparison and returns its value:

cmpl	$0, -4(%rbp)
sete	%al

The returning part is sete %al. In C's x86 calling conventions, return values 4 bytes or smaller (which includes int and bool) are returned via register %eax. %al is the lowest byte of %eax. So, the upper 3 bytes of %eax are left in an uncontrolled state.

Now in main():

call	f1
testl	%eax, %eax
je	.L2

This checks whether the whole of %eax is zero, because it thinks it's testing an int.

Adding an explicit function declaration changes main() to:

call	f1
testb	%al, %al
je	.L2

which is what we want.

Please compile with a command such as this one:

gcc -Wall -Wextra -Werror -std=gnu99 -o main.exe main.c

Output:

main.c: In function 'main':
main.c:14:5: error: implicit declaration of function 'f1' [-Werror=impl
icit-function-declaration]
     printf( f1() == true ? "true\n" : "false\n");
     ^
cc1.exe: all warnings being treated as errors

With such a message, you should know what to do to correct it.

Edit: After reading a (now deleted) comment, I tried to compile your code without the flags. Well, This led me to linker errors with no compiler warnings instead of compiler errors. And those linker errors are more difficult to understand, so even if -std-gnu99 is not necessary, please try to allways use at least -Wall -Werror it will save you a lot of pain in the ass.