Baffled - Dockerfile builds on all but one of our team's computers

General Discussions
1

We have a Dockerfile that works for most of our team’s computers, but not for one person.

docker build -t pq . runs to completion for all except one whose MacBook Pro is running macOS 10.15.7, and Docker Desktop 4.13.0.

The symptoms of the failure are described in the project issue but the major complaint is that cargo fails to compile libduckdb-sys v0.5.1 We can use cargo build to compile everything natively on that same MacBook.

My question: how do I even debug this? How can I get insight into what’s causing the error? Many thanks.

2

I tried to build it yesterday. My system is Docker Desktop 1.13 on macOS Monterey (M1 chiip). I got the same error message. Then I tried it in a multipass virtual machnie, but that froze at installing (or compiling??) a package called “arrow”.

To react some of the comments on GitHub:

The fact that you can the software on the host, does not really mean anyhing, macOS has not just its own packages, but its own kernel (Darwin). The only thing the host and Docker Desktop have in common is that if the host has an arm CPU, Docker Desktop will have that too, but it will run everything in a virtual machine.

When something is based on the kernel or anyhing close to it, you can have different result on different machines even if you run it in containers.

I also remember somthing that was an issue on Docker Hub, because there was a library I guess, that was not compatible with a specific Docker version so some builds failed. I didn’t have any problem, I just read about it.

Let’s say the build requires some device, for example GPU, or the build detects its environment and it builds differently in different environments. Since a container is not a full operating system, missing components, libraries can be a problem sometimes.

I know, it does not answer your main question.

Could you summarize for us where the build worked and whre it didn’t? I have read the comments on github, but it was not entirely clear to me.

I can test it on Mac, Linux and Windows 11

3

Hi! Thank you for the detailed response. I have asked my colleagues over at prql-query to send me their system info and will forward back to you.

For my own system: Building the prql-query Dockerfile FAILS on macOS 10.15.7 on Intel MacBook Pro, running Docker Desktop 4.13.0 and Docker version 20.10.20, build 9fdeb9c. It has trouble with libduckdb-sys as described in the original Github issue.

4

So it fails not just on one architecture. It wasn’t clear to me because based on the linked issue on GitHub, I thought it was related to the M1 chip.

Did it work on Docker Desktop on any architecture for anyone?

5

Thanks for checking back. I collected a couple other reports.

TL;DR: It works fine on Ubuntu; it works fine on Github; we have reports of working and failing on Mac M1 (yours and mine); I have never seen it work on an Intel Mac. All my tests use Docker Desktop; I can’t speak for the other reports.

I don’t know enough to say where this might lead us… Thanks again.

  • “Great — it succeeds for me on macOS 12.6, ARM M1 Max, Docker version 20.10.17, build 100c701.” link

  • “… it built fine on Github and builds on my Ubuntu machine as well.” link

  • Building the prql-query Dockerfile FAILS for me on both macOS 10.15.7 (Catalina) and macOS 12.6 (Monterey) on Intel MacBook Pro, running Docker Desktop 4.13.0 and Docker version 20.10.20, build 9fdeb9c. It the same problem with libduckdb-sys.

  • Building the prql-query Dockerfile FAILS for me on macOS 12.6 (Monterey) on M1 Macbook Air, running Docker Desktop 4.13.0 and Docker version 20.10.20, build 9fdeb9c. It the same problem with libduckdb-sys.

Update: For clarity, the failure I’m seeing is displayed in the original report. Click :arrow_forward: Details to see the full log. Look for error: failed to run custom build command for libduckdb-sys v0.5.1 (half-way down) to see what I believe is the error message.

6

Update: I used Docker Desktop on my Mac to increase Advanced → Resources → Memory from 8GBytes to 10GBytes, and the Dockerfile builds and operates as expected. Thanks!

2 Likes
7

I am surprised that increasing memory solved the error, but I am glad it helped. I didn’t have time to test it. Thanks for reporting back

8

I’m surprised too, especially that it requires > 8GBytes to compile. Thanks again.