What could cause a build to fail on Docker Hub when it succeeds locally?

Docker Hub
1

I have an automated build on Docker Hub (wch1/r-debug) for which the Docker build succeeds when I run it locally, but fails when I run it on Docker Hub. I suspect the problem has to do with some architecture differences between my local system and the build machine on Docker Hub. I hope someone out there can shed some light on this issue!

Some details: The Dockerfile builds R (the programming language) five times, each one with different settings. These various builds have different kinds of tooling to help debugging. Locally, all of the builds succeed. On Docker Hub, the first two succeed, and the third one fails. This third one enables AddressSanitizer and UndefinedBehaviorSanitizer when compiling with gcc. With this build, when it runs the configure script, it fails with:

checking whether we are cross compiling... 
configure: error: in `/tmp/r-source':

configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details

The environment variables it sets before running ./configure are:

export CXX="g++ -fsanitize=address,undefined,bounds-strict -fno-omit-frame-pointer"
export CFLAGS="${CFLAGS} -pedantic -fsanitize=address"
export FFLAGS="${CFLAGS}"
export FCFLAGS="${CFLAGS}"
export CXXFLAGS="${CFLAGS} -pedantic"
export MAIN_LDFLAGS="-fsanitize=address,undefined"

Is it possible that these flags cause the compiler to error out on Docker Hub, but not on my machine?

Info about my machine:

$ uname -a 
Linux userve 4.4.0-96-generic #119-Ubuntu SMP Tue Sep 12 14:59:54 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

$ docker version
Client:
 Version:      17.05.0-ce
 API version:  1.29
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:10:54 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.05.0-ce
 API version:  1.29 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:10:54 2017
 OS/Arch:      linux/amd64
 Experimental: false

Info about the build machine on Docker Hub, from the build log:

KernelVersion: 4.4.0-93-generic
Arch: amd64
BuildTime: 2017-08-17T22:50:04.828747906+00:00
ApiVersion: 1.30
Version: 17.06.1-ce
MinAPIVersion: 1.12
GitCommit: 874a737
Os: linux
GoVersion: go1.8.3

Thanks in advance for any help!

2

i’m not quite sure, but I think the error is right there…

--->cannot run C compiled programs (on docker hub machine)
--->See `config.log' for more details

on YOUR machine you CAN run that pgm /tmp/r-source

if that is an error in the config source, then you need to fix it… if you need that for some reason… then
you will have to find alternatives

3

There is another automated build that I know of on Docker Hub which tries to build R with similar settings. It worked in this build (https://hub.docker.com/r/rocker/r-devel-san/builds/bkdkwah2cdutblfsbks8zve/) but stopped working in this build https://hub.docker.com/r/rocker/r-devel-san/builds/bnwbhf2kzszgxrqazemsh6d/), even though there were no changes to the Dockerfile.

Here’s system information from the working build:

KernelVersion: 4.4.0-79-generic
Arch: amd64
BuildTime: 2017-03-28T19:26:53.326478373+00:00
ApiVersion: 1.27
Version: 17.03.1-ee-2
MinAPIVersion: 1.12
GitCommit: ad495cb
Os: linux
GoVersion: go1.7.5

And from the non-working build:

KernelVersion: 4.4.0-93-generic
Arch: amd64
BuildTime: 2017-08-17T22:50:04.828747906+00:00
ApiVersion: 1.30
Version: 17.06.1-ce
MinAPIVersion: 1.12
GitCommit: 874a737
Os: linux
GoVersion: go1.8.3

Thanks in advance for any help!

4

I was able to modify the build script to print out the config.log after it failed. The error was:

configure:6340: gcc -o conftest -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -O0 -Wall -pedantic -fsanitize=address -I/usr/local/include -L/usr/local/lib conftest.c  >&5
configure:6344: $? = 0
configure:6351: ./conftest
==311==Shadow memory range interleaves with an existing memory mapping. ASan cannot proceed correctly. ABORTING.
==311==ASan shadow was supposed to be located in the [0x00007fff7000-0x10007fff7fff] range.

When I searched for that error message, it looks like it’s probably due to a kernel bug: https://github.com/google/sanitizers/issues/837

According to this comment, the bug is present in Ubuntu kernel 4.4.0-93, which appears to be the version used on the failed Docker Hub builds. It also says that a workaround is to use the -no-pie linker flag, so I’ll try that for now. Once the Docker Hub build machines are updated, I’ll be able to remove that.