Getting "Illegal instruction" when I run my program inside container

I’m building an image on Ubuntu 16.04 (4.10.0-33-generic #37~16.04.1-Ubuntu, stretch) using C++. Then using docker (version 17.12.0-cd, build c97c6d6) to build an image. It runs fine on this machine. It also runs fine in Windows 10 Pro with docker version 17.09-1-ce, build 19e2cf6).

The lscpu for the build machine is:

Architecture:          x86_64                                                                                    
CPU op-mode(s):        32-bit, 64-bit                                                                            
Byte Order:            Little Endian                                                                             
CPU(s):                4                                                                                         
On-line CPU(s) list:   0-3                                                                                       
Thread(s) per core:    2                                                                                         
Core(s) per socket:    2                                                                                         
Socket(s):             1                                                                                         
NUMA node(s):          1                                                                                         
Vendor ID:             GenuineIntel                                                                              
CPU family:            6                                                                                         
Model:                 78                                                                                        
Model name:            Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz                                                  
Stepping:              3                                                                                         
CPU MHz:               447.924                                                                                   
CPU max MHz:           3400.0000                                                                                 
CPU min MHz:           400.0000                                                                                  
BogoMIPS:              5616.00                                                                                   
Virtualization:        VT-x                                                                                      
L1d cache:             32K                                                                                       
L1i cache:             32K                                                                                       
L2 cache:              256K                                                                                      
L3 cache:              4096K                                                                                     
NUMA node0 CPU(s):     0-3                                                                                       
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp                                                                                           

Then I loaded it onto Debian 7.11 (3.16.0-0.bpo.4-amd64 #1 SMP Debian, wheezy) using the same version of docker. When I run my program, it immediately stops after printing the message “Illegal instruction”.

The lscpu for this machine is:

Architecture:          x86_64         
CPU op-mode(s):        32-bit, 64-bit 
Byte Order:            Little Endian  
CPU(s):                2              
On-line CPU(s) list:   0,1            
Thread(s) per core:    1              
Core(s) per socket:    2              
Socket(s):             1              
NUMA node(s):          1              
Vendor ID:             GenuineIntel   
CPU family:            6              
Model:                 55             
Stepping:              9              
CPU MHz:               533.200        
BogoMIPS:              2664.00        
Virtualization:        VT-x           
L1d cache:             24K            
L1i cache:             32K            
L2 cache:              512K           
NUMA node0 CPU(s):     0,1            

Other programs in this same image run.

This program contains my own code plus RocksDB, gRPC, Boost and another proprietary library.

I’m not sure where to look. Is there a compiler option is should use to generate instructions for older CPUs?

Anything else I should look at?

I eventually solved this by recompiling RockdDB using PORTABLE=1 make static_lib.

1 Like

I encounter the same pattern of the issue. May I ask how do you locate the issue? Like how do you find out it locates in the lib of RockdDB