Unable to delete directory in container: rmdir: failed to remove '<DIR_NAME>' Device or resource busy

General Discussions
1

I have a dockerized application that has an entrypoint script that goes in to the container and deletes the automatically created code directory to create a link of the same name of the application directory. But every time when I run:
docker-compose up -d

The docker container keep restarting and on looking at the logs:
docker logs <container_name>

I get the error:
rmdir: failed to remove '/code': Device or resource busy

Any idea as to how to resolve this issue and what is going on?

2

I don’t think you will be allowed to erase the directory, but you might be able to erase the contents inside.
If you could show the Dockerfile and docker-compose file that might help figure out the issue.

4

Dockerfile:

FROM <USER_NAME>/al-node:latest as node

# Clone app code into /code
ARG AL_VERSION=3.0.4
ARG GIT_BRANCH
ARG GIT_URL=<URL_HERE>
ENV AL_VERSION="${AL_VERSION}"
RUN echo "git branch: ${GIT_BRANCH}"
RUN apk add --no-cache git \
    && git clone --depth 1 -b "${GIT_BRANCH}" "${GIT_URL}" code \
    && echo "Cache-bust: ${AL_VERSION}"

# ---

FROM buildpack-deps:bionic

LABEL maintainer=<NAME>
WORKDIR /code
ARG AL_VERSION=3.0.4
ENV PYTHONUNBUFFERED=1 LANG=C.UTF-8 AL_VERSION="${AL_VERSION}"

# include  packages
RUN apt-get clean \
    && apt-get update \
    && DEBIAN_FRONTEND=noninteractive apt-get install -y -q --no-install-recommends \
        build-essential \
        g++ \
        gfortran \
        python3-all-dev \
        python3-pip \
        python3-setuptools \
        python3-wheel \
    && apt-get autoremove -y \
    && ln -s /usr/bin/python3 /usr/local/bin/python \
    && ln -s /usr/bin/pip3 /usr/local/bin/pip \
    && pip install --upgrade pip setuptools wheel \
    && pip install --no-cache-dir virtualenv \
    && rm -rf /var/lib/apt/lists/*

COPY requirements.txt /tmp/

# Install packages
RUN pip install -r /tmp/requirements.txt

#  Tini
# Keep for now to maintain backward-compatibility; better to launch with docker run --init
ENV TINI_VERSION v0.16.1
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /tini
RUN chmod +x /tini

# entrypoint scripts
COPY entrypoint.sh /usr/local/bin

# Copy in code from node image
COPY --from=node /code /usr/local/al

# Copy in AL python code
#RUN git clone --depth 1 -b master <GIT_URL> /usr/local/al

ENTRYPOINT ["/tini", "--", "/usr/local/bin/entrypoint.sh"]
CMD ["--help"]

In the entrypoint.sh there a script like this:

if [ ! -x /code/manage.py ]; then
    # first get rid of auto-created /code directory in Dockerfile
    cd / && rmdir /code  <---------------------- FAILS HERE!!
    ln -sv /usr/local/al /code

On running docker-compose the containers come up but fails to remove the directory giving the error:
rmdir: failed to remove ‘/code’: Device or resource busy

I don’t want to give the compose file as its sensitive. Please ask me any questions that you might have. Thanks

5

Thanks. So explain why you can’t remove the /code directory in your Dockerfile after you COPY it to /usr/local/al

6

Hi Gary,

  1. I am cloning the code to the /code directory on the container and then copying the code there and the node image to /usr/local/al.

  2. Once copied I am creating a soft link of the /usr/locl/al directory as /code.

  3. In order to that I need to delete the existing /code before I can create a link, otherwise it throws an error that fiel already exists, cannot create link.

I am currently getting the error when I am trying to remove the /code directory.

Many thanks for your time!

7

I’ve “simplified” your Dockerfile below to help understand your build workflow.
You have a multi stage build with 2 images: <USER_NAME>/al-node:latest and buildpack-deps:bionic. I added a RUN rm -rf /code statement after the COPY statement. The directory gets removed during the build of the image. You can then remove the erase of the directory in the entrypoint.sh script.

FROM <USER_NAME>/al-node:latest as node
********
FROM buildpack-deps:bionic
********
COPY --from=node /code /usr/local/al
RUN rm -rf /code
ENTRYPOINT ["/tini", “–”, “/usr/local/bin/entrypoint.sh”]
CMD ["–help"]
8

I think that might work. Let me get back to you, just in case.

9

I’m also thinking you could create the link in the Dockerfile and remove that too from your entrypoint script.

FROM <USER_NAME>/al-node:latest as node
********
FROM buildpack-deps:bionic
********
COPY --from=node /code /usr/local/al
RUN rm -rf /code && ln -sv /usr/local/al /code

ENTRYPOINT ["/tini", “–”, “/usr/local/bin/entrypoint.sh”]
CMD ["–help"]
10

It doesn’t help. I removed the related code from entrypoint.sh after adding them in the Dockerfile as above but it fails in this script in the entrypoint.sh:

if [ ! -f /code/al/settings/local.py ]; then
    output "Creating local.py from example ..."
    cp /code/al/settings/local.py-example /code/al/settings/local.py <---------------- FAILS HERE!!
fi

Error: cp: cannot stat '/code/al/settings/local.py-example': No such file or directory

I checked the contents of the code directory and it shows like this:

lrwxrwxrwx  1 root root   20 Feb  6 01:12 al -> /usr/local/al
drwxr-xr-x  2 root root 4.0K Feb  6 01:12 .
drwxr-xr-x 13 root root 4.0K Feb  6 04:19 ..

My question is do you guys know of this issue with creating soft links in Docker containers. Also this same code works when I run on my mac and builds just fine. This fails when I run the same code on a Debian server.

Any inputs?

11

Also to add to that I do not know why rmdir or rm-r is failing in the containers. As I said it works for containers spun up on a Mac but not when done on a Debian server. Thanks

12

Hi.

Couple of things.

Docker for MacOS runs Docker in a linux virtual box as Docker does not running natively on MacOS.
So that is “different” then running Docker on Linux. But I think the latest error you posted would happen on MacOS and Linux.

I looked at the latest error you are getting and it does not make sense to me.

if [ ! -f /code/al/settings/local.py ]; then
output “Creating local.py from example 
”
cp /code/al/settings/local.py-example /code/al/settings/local.py <---------------- FAILS HERE!!
fi

Error: cp: cannot stat ‘/code/arrowland/settings/local.py-example’: No such file or directory

The copy statement that you say is failing is:
cp /code/al/settings/local.py-example /code/al/settings/local.py
But the error shows:
Error: cp: cannot stat ‘/code/arrowland/settings/local.py-example’: No such file or directory
I don’t see the subdirectory arrowland in the cp statement in your entrypoint.sh script.

I recommend that you run your container and bring up a command prompt inside of it. Then display the contents of the /code directory and the contents of the /usr/local/al directory.

If I could see the entire contents of your entrypoint.sh script and the contents of the /code directory and the contents of the /usr/local/al at containter start time I might be able to figure out the problem.

13

Sorry for the confusion. I corrected the path, that was a typo.

My container keeps restarting because of the error and that makes it difficult for me to debug. It is a very weird. This is a project under development and I might run into proprietary issues so I am not sure i can share the scripts, although I understand your concern.Thanks.

14

Run your image in a container with a docker container run and bring up a bash shell inside the container.
Then once inside the container, run your entrypoint.sh script.

docker container run -ityour-docker-image bash

15

I tried that and it worked inside the container but didn’t work when I used my docker compose up.

SOLUTION: Upon investigation I found out that I was mounting a local copy in the override file for one of my services in the same location. Removed that and worked like a charm!! Thanks a lot Gary, you have been very helpful!

1 Like
16

Awesome! Glad you got it working.