Docker stops responding on MacOS, error on Arch Linux while building image

demensdeum · April 2, 2024, 8:04pm

Docker stops responding on MacOS, error on Arch Linux while building image.

OS: MacOS Sonoma 14.4.1 and Arch Linux
App version:
 Cloud integration: v1.0.35+desktop.11
 Version:           25.0.3
 API version:       1.44
 Go version:        go1.21.6
 Git commit:        4debf41
 Built:             Tue Feb  6 21:13:26 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Steps to reproduce:

git clone https://github.com/demensdeum/Masonry-AR-Docker.git
cd Masonry-AR-Docker
docker buildx build .

wait some time
stuck at “CXX Magick++/lib/libMagick___7_Q16HDRI_la-TypeMetric.lo”
docker ps not responding
docker images not responding

If I remove ImageMagick build from Dockerfile and move it to terminal, then ImageMagick build ends with error:
ERRO[0033] error waiting for container: unexpected EOF
and after that docker ps, docker images did not respond

rimelek · April 2, 2024, 11:30pm

I don’t understand. How is Arch Linux in the title related to the issue?

demensdeum · April 3, 2024, 4:55am

On Arch Linux I got error in terminal instantly after calling docker ps or docker images after stuck, on MacOS it’s just silent

demensdeum · April 3, 2024, 11:52am

For 26.0.0 version on MacOS I got:
ERROR: failed to receive status: rpc error: code = Unavailable desc = error reading from server: EOF

while “docker-buildx build .”
I think there is bug in builder, similar to “failed to receive status: rpc error: code = Unavailable desc = error reading from server: EOF · Issue #2064 · docker/buildx · GitHub”

rimelek · April 3, 2024, 7:01pm

You mean Docker CE v25.0.0 in Docker Desktop for Mac? Because as far as I know, the latest version on macOS is Docker Desktop 4.28 which has Docker CE 25.0.3 in it. but docker ps not responding doesn’t seem to be related.

Regarding the error on macOS, “waiting for container: unexpecteed EOF” could be that the API server can’t respond properly because the build requires too much resources. That would explain why docker ps can’t respond either. Did you try to watch the bottom of the Docker Desktop window while the build runs to see the used resources? You could also optimize your Dockerfile as this doing an upgrade like this is highly discouraged

github.com

demensdeum/Masonry-AR-Docker/blob/d67c95b3efe69bac9ad4c8b4060dc644da72423c/Dockerfile#L4C1-L4C19


      
          RUN apt -y upgrade

It is not a vitual machine. Install what you need and don’t touch dependencies you don’t unless you know about a serious security risk. Otherwise you can actually introduce new security holes or in some cases you can upgrade almost every package which just makes your image much bigger. I could mention more problems, but this topic is not about making a perfect Dockerfile so I won’t do it.

demensdeum · April 3, 2024, 7:24pm

I tried 26.0.0 for MacOS version from brew.
But first I reproduced by Docker Desktop.

Problem disappears after replacing lines with ImageMagick 7 compilation to compilation from another repository:

I will check resources + play with Dockerfile, and report if I found anything more, aside from imei workaround.

rimelek · April 3, 2024, 8:40pm

How does that work? Can you share the command? macOS doesn’T support containers, so you need the Desktop. Did you install the Docker CE 26.0.0 clint only?

demensdeum · April 3, 2024, 10:34pm

Yes here commands:

brew install docker
brew install docker-buildx
brew install colima
colima start

After that

docker --version   
Docker version 26.0.0, build 2ae903e86c

rimelek · April 4, 2024, 4:08pm

That is the client. It can be useful when you don’t want to run Docker on your machine just connect to a remote machine, but the client comes with Docker Desktop. If you want to use brew, brew install docker-desktop should work probably. Since this is not the officially ecommended way, I always forget A different client could cause hanging docker ps, but then it would never work not just after trying to build an image, so the problem will be on the Docker Daemon side in the virtual machine. Just to avoid other problems caused by the client, I recommend deleting the “docker” brew package.

demensdeum · April 4, 2024, 5:24pm

This does not work without colima, and colima is “container runtimes on macOS”, I can assume that they run linux in vm, and then docker on the top of this linux-vm.
I got original problem without colima, or other client, just vanilla Docker Desktop.

rimelek · April 4, 2024, 5:37pm

You mean the docker command from brew? Yes colima runs a virtual machine, but the client is just a client. It could work with Docker Desktop or any daemon since the contexts are configured in $HOME/.docker. I ran brew install docker and since I already had the client from Docker Desktop, brew didn’t replace my client. It was only kept in /opt/homebrew. I don’t know what Docker Desktp would do if it is installed after the other client.

demensdeum · April 4, 2024, 6:19pm

Can’t reproduce problem on Windows 11 with Docker Desktop:
docker --version
Docker version 25.0.3, build 4debf41

Reproducable only on Arch Linux (x86-64) and MacOS Sonoma 14.4.1 with M2 CPU

I think this is race condition bug in buildx (different thread libs?), added issue into their bug tracker

github.com/docker/buildx

ERROR: failed to receive status: rpc error: code = Unavailable desc = error reading from server: EOF

opened 12:40PM - 03 Apr 24 UTC

demensdeum

status/triage

### Contributing guidelines - [X] I've read the [contributing guidelines](https…://github.com/docker/buildx/blob/master/.github/CONTRIBUTING.md) and wholeheartedly agree ### I've found a bug and checked that ... - [X] ... the documentation does not mention anything about my problem - [X] ... there are no open or closed issues that are related to my problem ### Description git clone https://github.com/demensdeum/Masonry-AR-Docker.git cd Masonry-AR-Docker docker buildx build . <after some time> ERROR: failed to receive status: rpc error: code = Unavailable desc = error reading from server: EOF ### Expected behaviour Image build success ### Actual behaviour Image build failed ### Buildx version github.com/docker/buildx v0.12.1-desktop.4 6996841df2f61988c2794d84d33205368f96c317 ### Docker info ```text Client: Version: 25.0.3 Context: desktop-linux Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.12.1-desktop.4 Path: /Users/iliaprokhorov/.docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: v2.24.6-desktop.1 Path: /Users/iliaprokhorov/.docker/cli-plugins/docker-compose debug: Get a shell into any image or container. (Docker Inc.) Version: 0.0.24 Path: /Users/iliaprokhorov/.docker/cli-plugins/docker-debug dev: Docker Dev Environments (Docker Inc.) Version: v0.1.0 Path: /Users/iliaprokhorov/.docker/cli-plugins/docker-dev extension: Manages Docker extensions (Docker Inc.) Version: v0.2.22 Path: /Users/iliaprokhorov/.docker/cli-plugins/docker-extension feedback: Provide feedback, right in your terminal! (Docker Inc.) Version: v1.0.4 Path: /Users/iliaprokhorov/.docker/cli-plugins/docker-feedback init: Creates Docker-related starter files for your project (Docker Inc.) Version: v1.0.1 Path: /Users/iliaprokhorov/.docker/cli-plugins/docker-init sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.) Version: 0.6.0 Path: /Users/iliaprokhorov/.docker/cli-plugins/docker-sbom scout: Docker Scout (Docker Inc.) Version: v1.5.0 Path: /Users/iliaprokhorov/.docker/cli-plugins/docker-scout Server: Containers: 1 Running: 0 Paused: 0 Stopped: 1 Images: 4 Server Version: 25.0.3 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Using metacopy: false Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb runc version: v1.1.12-0-g51d5e94 init version: de40ad0 Security Options: seccomp Profile: unconfined cgroupns Kernel Version: 6.6.16-linuxkit Operating System: Docker Desktop OSType: linux Architecture: aarch64 CPUs: 8 Total Memory: 3.831GiB Name: docker-desktop ID: 8a77388d-38da-47e1-a973-63acdd60655c Docker Root Dir: /var/lib/docker Debug Mode: false HTTP Proxy: http.docker.internal:3128 HTTPS Proxy: http.docker.internal:3128 No Proxy: hubproxy.docker.internal Username: demensdeum Experimental: false Insecure Registries: hubproxy.docker.internal:5555 127.0.0.0/8 Live Restore Enabled: false ``` ### Builders list ```text NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS default docker default default running v0.12.5 linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6 desktop-linux * docker desktop-linux desktop-linux running v0.12.5 linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6 ``` ### Configuration FROM ubuntu RUN apt update RUN apt -y upgrade RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get -y install tzdata RUN apt install -y wget RUN apt install -y apache2 RUN apt install -y php RUN apt install -y libapache2-mod-php RUN apt install -y php-mysql RUN apt install -y mariadb-server RUN apt install -y git RUN apt install -y unzip RUN apt install -y python3 RUN apt install -y curl RUN apt install -y composer RUN apt install -y php-pear RUN apt-get install -y php-dev RUN pecl install ast RUN ln -s /bin/python3 /bin/python RUN apt-get update && apt-get install -y wget && \ apt-get install -y autoconf pkg-config RUN apt-get update && apt-get install -y wget && \ apt-get install -y build-essential curl libpng-dev && \ wget https://github.com/ImageMagick/ImageMagick/archive/refs/tags/7.1.0-31.tar.gz && \ tar xzf 7.1.0-31.tar.gz && \ rm 7.1.0-31.tar.gz && \ apt-get clean && \ apt-get autoremove RUN sh ./ImageMagick-7.1.0-31/configure --prefix=/usr/local --with-bzlib=yes --with-fontconfig=yes --with-freetype=yes --with-gslib=yes --with-gvc=yes --with-jpeg=yes --with-jp2=yes --with-png=yes --with-tiff=yes --with-xml=yes --with-gs-font-dir=yes && \ make -j && make install && ldconfig /usr/local/lib/ RUN rm /var/www/html/index.html RUN a2enmod ssl RUN mkdir ssl RUN apt-get update && \ apt-get install -y openssl && \ openssl genrsa -des3 -passout pass:xxxx -out /ssl/server.pass.key 2048 && \ openssl rsa -passin pass:xxxx -in /ssl/server.pass.key -out /ssl/server.key && \ rm /ssl/server.pass.key && \ openssl req -new -key /ssl/server.key -out /ssl/server.csr \ -subj "/C=UK/ST=Warwickshire/L=Leamington/O=OrgName/OU=IT Department/CN=example.com" && \ openssl x509 -req -days 365 -in /ssl/server.csr -signkey /ssl/server.key -out /ssl/server.crt RUN rm /etc/apache2/sites-enabled/000-default.conf ADD ssl/000-default.conf /etc/apache2/sites-enabled/000-default.conf ADD script/installMasonryAR.sh / RUN rm /etc/php/8.1/cli/php.ini ADD php/php.ini /etc/php/8.1/cli/php.ini RUN apt install -y pip RUN apt-get install php-mysqli RUN echo "1" > /DockerImage ### Build logs ```text docker buildx build . [+] Building 49.5s (25/38) docker:desktop-linux => CACHED [ 7/34] RUN apt install -y php 0.0s => CACHED [ 8/34] RUN apt install -y libapache2-mod-php 0.0s => CACHED [ 9/34] RUN apt install -y php-mysql 0.0s => CACHED [10/34] RUN apt install -y mariadb-server 0.0s => CACHED [11/34] RUN apt install -y git 0.0s => CACHED [12/34] RUN apt install -y unzip 0.0s => CACHED [13/34] RUN apt install -y python3 0.0s => CACHED [14/34] RUN apt install -y curl 0.0s => CACHED [15/34] RUN apt install -y composer 0.0s => CACHED [16/34] RUN apt install -y php-pear 0.0s => CACHED [17/34] RUN apt-get install -y php-dev 0.0s => CACHED [18/34] RUN pecl install ast 0.0s => CACHED [19/34] RUN ln -s /bin/python3 /bin/python 0.0s => CACHED [20/34] RUN apt-get update && apt-get install -y wget && a 0.0s => CACHED [21/34] RUN apt-get update && apt-get install -y wget && a 0.0s => [22/34] RUN sh ./ImageMagick-7.1.0-31/configure --prefix=/usr/local 49.5s => => # CXX Magick++/lib/libMagick___7_Q16HDRI_la-ResourceLimits.lo => => # CXX Magick++/lib/libMagick___7_Q16HDRI_la-SecurityPolicy.lo => => # CXX Magick++/lib/libMagick___7_Q16HDRI_la-Statistic.lo => => # CXX Magick++/lib/libMagick___7_Q16HDRI_la-STL.lo => => # CXX Magick++/lib/libMagick___7_Q16HDRI_la-Thread.lo => => # CXX Magick++/lib/libMagick___7_Q16HDRI_la-TypeMetric.lo ERROR: failed to receive status: rpc error: code = Unavailable desc = error reading from server: EOF ``` ### Additional info Reproduces on Arch Linux machine and M2 MacBook