Docker Community Forums

Share and learn in the Docker community.

Multiple projects stopped building on Docker Hub, "Operation not permitted"

I have several projects linked to Github repositories that build via a Dockerfile in each project. This arrangement has been working fine, but all my builds started failing 100% of the time a while back. I’ve been trying to trace it and realized I didn’t really do anything and the issue it fails on is quite basic.

I’m using the archlinux base image and copying the source of my app to the container using something like:

COPY ./ /src

Next I move into that directory:

WORKDIR /src

Then I try to build the software. Normally this would involve running ./configure, but that dies with a weird error saying “Operation not permitted”. It turn out this isn’t just an issue for GNU Make, even ls can’t read the directory! Specifically the directory, not the stuff inside it. I can run RUN ls -l and get a list of files in the source that got copied over, but I can’t run RUN ls -ld to show the directory properties, it dies like this:

Step 17/28 : RUN ls -ald
---> Running in ec8f9f6c3604
ls: cannot access '.': Operation not permitted

Removing intermediate container ec8f9f6c3604
The command '/bin/sh -c ls -ald' returned a non-zero code: 2

I can run various other commands, but anything that tries to look at the directory itself dies like this. I can even create files in the directory.

Note these same Dockerfiles build just fine on my local system using docker build. What gives? What is different about building them on Docker Hub?

YES, thank god somebody else posted an issue with operation not permitted, i have been banging my head against a brick wall for 2 days now with this issue, i get issues during makepkg, if i build locally it builds fine!, started approx 4 days ago for me, snippet from the console output on docker hub:-

e[91m==> WARNING: Using existing $srcdir/ tree
e[0m
==> Starting build()...
e[91mls: cannot access '.': Operation not permitted
configure: error: working directory cannot be determined
e[0m
e[91m==> ERROR: A failure occurred in build().
Error making: makemkv

Of note is that i am also using Arch Linux here as my base OS, so its not improbable for this to be a Arch Linux issue and not a Docker hub issue - still if that were the case then why can i build the image locally?!.

I’ve opened an issue report upstream on the Arch Linux docker repository which supposedly holds the tooling used to generate the image in the first place.

Thanks for the reply, i will watch the issue with interest.

We’ve traced this as far as being related to the latest GNU coreutils package. Everything is fine unless you load the latest release of that package into the container, then directories become un-stat-able. Again this only surfaces when building on Docker Hub, building anywhere else seems fine.

Does anybody have this problem on other distros? Are there even any that have brand new coreutils?

Docker folks does this ring any bells for you?

Just a follow up for anybody hitting the same issue:

  1. This is happening because the Docker Hub hosts run old kernels that are not compatible with the latest GNU coreutils, specifically the statx system calls.

  2. The Docker host kernels don’t properly announce their version and hence alert coreutils that they can’t make statx calls, something they would normally gracefully work around.

  3. This is not just Arch Linux, all distributions with new packages are hitting the same issue. Recent builds of Fedora have the same problem, SUSE Tumbleweed, etc.

  4. This isn’t just Docker Hub, Travis is also causing a similar issue.

1 Like

I am still getting these errors for OpenSuSE Tumbleweed and Fedora Rawhide. Does anyone know when this is going to b e fixed? Any workaround that I can use? I need containers from bleeding edge distributions.

No clue. Docker themselves haven’t so much as acknowledged that there is an issue that I know of. Fedora and Arch Linux have been throwing around hacks to keep coreutils held back. The GNU folks didn’t do anything wrong, their implementation and backwards compatibility shim seem to be okay, this is an issue –as I understand it– with the hypervisor reporting functionality but not delivering, especially when paired with old host kernels and/or security options.