/lib64/libc.so.6: version `GLIBC_2.32' not found

Open Source Projects Compose
, ,
1

I have a Quarkus application that is used to build a native-micro container image. The image is built in a GitHub Action. However, when I run it on my Windows laptop using docker-compose I receive the following error.

./application: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by ./application)
./application: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by ./application)

It seems the reason is the difference in build and run environments w.r.t. glibc as described at aws lambda - /lib64/libc.so.6: version `GLIBC_2.32' not found - Stack Overflow. There is also a reference to avoid this problem in Quarkus - aws lambda - /lib64/libc.so.6: version `GLIBC_2.32' not found - Stack Overflow. But, wasn’t the promise of containers to avoid “works on my laptop” reason?

The container image is built in a GitHub Action where the following is the output of the step for building the container image. The Docker Desktop on my Windows laptop is v4.21.1.

docker version

 /usr/bin/docker version
  Client:
   Version:           20.10.25+azure-2
   API version:       1.41
    Network: bridge host ipvlan macvlan null overlay
    Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
   Swarm: inactive
   Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
   Default Runtime: runc
   Init Binary: docker-init
   containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
   runc version: 82f18fe0e44a59034f3e1f45e475fa5636e539aa
   init version: 
   Security Options:
    apparmor
    seccomp
     Profile: default
    cgroupns
   Kernel Version: 5.15.0-1041-azure
   Operating System: Ubuntu 22.04.2 LTS
   OSType: linux
   Architecture: x86_64
   CPUs: 2
   Total Memory: 6.769GiB
   Name: fv-az749-417
   ID: 3VSB:NXIS:24L2:M6PQ:TJO2:XRKH:EU3Y:O5W5:TBXJ:WCJP:B7JM:JHCV
   Docker Root Dir: /var/lib/docker
   Debug Mode: false
   Username: githubactions
   Registry: https://index.docker.io/v1/
   Labels:
   Experimental: false
   Insecure Registries:
    127.0.0.0/8
   Live Restore Enabled: false

buildx information

  /usr/bin/docker buildx version
  github.com/docker/buildx 0.11.2+azure-1 9872040b6626fb7d87ef7296fd5b832e8cc2ad17
2

Yes and no. Containers will not solve all the problems, but if you build and run the application in the same container or at least based on the same image, you shouldn’t have problem with the version of the libraries. In some cases it is also important that the Docker version is compatible with the operating system in the container which sends system calls through Docker.

So based on the shared sources I guess there is a difference between the image on your Windows and the one available in GitHub Actions. How do you refer to the base image?

3

The image is pushed to ghcr.io by GitHub Actuon. The same image (:latest) is pulled and ran on my Windows laptop. I can verify with SHA sums too, but, pretty sure the images are the same.

Is the issue due to difderent versions of docker runtime? Or, way the binary image is packaged as container?

4

So, the issue is not with Quarkus base image. Instead with container runtime (docker, podman, containerd, etc.) Or, with the host where container runtime is installed?

The issue being glibc version mismatch.

5

I just mentioned the possibility. I don’t think it is what happened in your case.

6

From where does the application expect the libraries to be provisioned? Base image, container runtime, docker host?

7

Sorry for the late answer. The application itself of course uses libraries that are in the container which comes from the base image. Issues with the docker version matter only when there is a system call sent by an app from inside the container and the system call is not compatible with Docker. I’m not entirely sure how it works. I had the system call issue once and never after but I read about issues from others too. I alsou encountered other issues as well like when an image could be built only on one server and other servers could not run the container or rebuild the image until some changis in the application’s config. I never found out why it happened I only suspected some CPU specific dependency.

Unfortunately I don’t know the answer to your problem, but it looks like someone else just had a similar issue:

8

I ran the ldd --version command in the GitHub Action pipeline to understand the glibc version in the runner. I also ran the same command for the Quarkus image for native-micro. The glibc version in Quarkus image turned out to be older - hence, the error.

Therefore, at the moment, I have two choices.

Identify builder machine proportionate to the glibc version in Quarkus base image
Identify a build machine with the right level of operating system and therefore, glibc version. The right level here means, the same as the one used in the Quarkus base image. This is not easy with the GitHub Actions as the matching glibc will require “old” OS that are not found (see here as an example - DistroWatch.com: CentOS ) in GitHub Actions. The alternative is to use a self-hosted runner but, even there, the required OS might be a challenge for various IaaS providers.

Identify alternative micro base image that has a recent glibc version
Use an alternative base image with a “recent” glibc version that can run the generated binary. In other words, rather than looking for right build machine, I am looking for a base image where I can run my binary that was generated on a “recent” build machine.

I am more hopeful of second one. I am now looking for the “tiniest” image with the latest glibc. Any quick thoughts?

Also following up here.

9

Found resolution from the SO link java - Alternatives to Quarkus base images for native-micro - Stack Overflow where, the guidance is to copy glibc related libraries from UBI base image into UBI micro image. The modified Dockerfile now looks as follows.

FROM registry.access.redhat.com/ubi9-minimal:9.2 AS ubi
# -- Stage scratch
FROM registry.access.redhat.com/ubi9-micro:9.2 AS scratch
# -- Final Stage
FROM scratch
COPY --from=ubi /usr/lib64/libgcc_s.so.1 /usr/lib64/libgcc_s.so.1
COPY --from=ubi /usr/lib64/libstdc++.so.6 /usr/lib64/libstdc++.so.6
COPY --from=ubi /usr/lib64/libz.so.1 /usr/lib64/libz.so.1

WORKDIR /work/
RUN chown 1001 /work \
    && chmod "g+rwX" /work \
    && chown 1001:root /work
COPY --chown=1001:root target/*-runner /work/application

EXPOSE 8080
USER 1001

CMD ["./application", "-Dquarkus.http.host=0.0.0.0"]

I just hope the “indiscriminate” copy of glibc won’t spring up other issues!