Invalid instruction when starting a minimal OpenMP program

Question

This minimal OpenMP program

#include <omp.h> int main() { #pragma omp parallel sections { #pragma omp section { while(1) {} } #pragma omp section { while(1) {} } } } 

will create this error when compiling and starting with gcc test.c -fopenmp :

 Illegal instruction (core dumped) 

When I change one of the loops with

  int i=1; while(i++) {} 

or any other condition that it compiles and runs without errors. It seems that 1 as a condition of the loop in different threads causes some strange behavior. Why?

edit: I am using gcc 4.6.3

edit: This is a bug in gcc and was introduced as a bug 54017 by gcc developers.

Answer 1

This is apparently a bug in GCC. GCC implements OpenMP sections using the GOMP_sections_start() procedure from libgomp , which returns the identifier of section 1 that the calling thread should execute, or 0 if all work items have been distributed. Basically the converted code should look like this:

 main._omp_fn.0 (void * .omp_data_i) { unsigned int .section.1; .section.1 = GOMP_sections_start(2); L0: switch (.section.1) { case 0: // No more sections to run, exit goto L2; case 1: // Do section 1 while (1) {} goto L1; case 2: // Do section 2 while (1) {} goto L1; default: // Impossible section value, possible error in libgomp __builtin_trap(); } L1: .section.1 = GOMP_sections_next(); goto L0; L2: GOMP_sections_end_nowait(); return; } 

It happens that in your case both the default and 0 events lead to __builtin_trap() . __builtin_trap() is the built-in GCC that should interrupt your program abnormally, and on x86 it issues the ud2 command, which causes the processor to bark with the illegal exception of the operation code. Usually it is placed in places where the code should never be executed, for example. all possible correct return values ​​from GOMP_sections_start() and GOMP_sections_next() should be covered by cases in the switch, and if the default value is reached (signaling a possible error in libgomp ), it should fail, and you will complain to the developers :)

Edit: This is definitely not expected OpenMP behavior, and it does not happen with icc or suncc . I sent Bug 54017 to GCC Bugzilla.

Edit 2: I updated the text to more accurately reflect what GCC should produce. It seems that GCC is getting the wrong idea of ​​the control flow in the parallel region and is doing some “optimizations” that spoil the code generation even more.

Answer 2

SIGILL was created because there is an illegal ud2 / ud2a instruction. According to http://asm.inightmare.org/opcodelst/index.php?op=UD2 :

This instruction called #UD. Intel guaranteed that in the future Intel CPU this instruction will call #UD. Of course, all previous processors (186+) called #UD in this opcode. This instruction is used by writer software to test the #UD exception service routine.

Take a peek inside:

 $ gcc-4.6.2 -fopenmp omp.c -o omp $ gdb ./omp ... (gdb) r Program received signal SIGILL, Illegal instruction. ... 0x08048544 in main._omp_fn.0 () (gdb) x/i $pc 0x8048544 <main._omp_fn.0+28>: ud2a (gdb) disassemble Dump of assembler code for function main._omp_fn.0: 0x08048528 <main._omp_fn.0+0>: push %ebp 0x08048529 <main._omp_fn.0+1>: mov %esp,%ebp 0x0804852b <main._omp_fn.0+3>: sub $0x18,%esp 0x0804852e <main._omp_fn.0+6>: movl $0x2,(%esp) 0x08048535 <main._omp_fn.0+13>: call 0x80483f0 < GOMP_sections_start@plt > 0x0804853a <main._omp_fn.0+18>: cmp $0x1,%eax 0x0804853d <main._omp_fn.0+21>: je 0x8048548 <main._omp_fn.0+32> 0x0804853f <main._omp_fn.0+23>: cmp $0x2,%eax 0x08048542 <main._omp_fn.0+26>: je 0x8048546 <main._omp_fn.0+30> 0x08048544 <main._omp_fn.0+28>: ud2a 0x08048546 <main._omp_fn.0+30>: jmp 0x8048546 <main._omp_fn.0+30> 0x08048548 <main._omp_fn.0+32>: jmp 0x8048548 <main._omp_fn.0+32> End of assembler dump. 

The assembler file already has ud2a:

 $ gcc-4.6.2 -fopenmp omp.c -o omp.S -S; cat omp.S main._omp_fn.0: .LFB1: pushl %ebp .LCFI4: movl %esp, %ebp .LCFI5: subl $24, %esp .LCFI6: movl $2, (%esp) call GOMP_sections_start cmpl $1, %eax je .L4 cmpl $2, %eax je .L5 .value 0x0b0f 

.value 0xb0f is the ud2a code

After verifying that ud2a was inserted with gcc intent (in the early stages of openmp), I tried to understand the code. The main._omp_fn.0 function is the body of the parallel code; it will call _GOMP_sections_start and _GOMP_sections_start its return code. If the code is 1, then we move on to one infinite loop; if it is 2, go into the second infinite loop. But in another case, ud2a will be executed. (I don’t know why, but according to Hristo Iliev, this is GCC Error 54017. )

I think this test is good to check how many processor cores there are. By default, the GCC openmp library (libgomp) will start a thread for each processor core on your system (in my case there were 4 threads). And the sections will be selected in order: the first section for the first stream, the second section - the 2nd stream, etc.

There is no SIGILL if I run the program on 1 or 2 processors (the tasket option is the processor mask in hexadecimal format):

  $ taskset 3 ./omp ... running on cpu0 and cpu1 ... $ taskset 1 ./omp ... running first loop on cpu0; then run second loop on cpu0...