If it matters, I am using Docker version 19.03.1 build 74b1e89, Docker Desktop (community) 2.1.0.1, and macOS 10.14.6 (Mojave).
My dockerfile contains the following lines:
RUN cd /home/renderaccount && tar -xzvf main.tar.gz
RUN mv /home/renderaccount/main /var/lib/postgresql/10/main
RUN cd /var/lib/postgresql/10/ && chown -R postgres:postgres main
when executing the last line, I get the error message:
Error processing tar file(exit status 1): write /var/lib/postgresql/10/main/base/16385/4468958.2: no space left on device
If I understand the docker system correctly, during a build, an intermediate layer is constructed for each command. The docker system apparently decides, for each command, how big that layer needs to be for the command to be executed successfully. Here is a partial layer history of the failed build:
$ docker history 4fd36f61aace
IMAGE CREATED CREATED BY SIZE COMMENT
4fd36f61aace 12 minutes ago /bin/sh -c mv /home/renderaccount/main /var/… 48.1GB
a5b7b9ffe84d 29 minutes ago /bin/sh -c lsof | grep deleted || true 0B
d36f3448ddee 29 minutes ago /bin/sh -c df -i 0B
905e2bfa465b 29 minutes ago /bin/sh -c df -h 0B
648ee7f2b7a0 31 minutes ago /bin/sh -c cd /home/renderaccount && tar -xz… 48.1GB
8161755c565d 2 hours ago /bin/sh -c lsof | grep deleted || true 0B
2ceb6441f2bd 2 hours ago /bin/sh -c df -i 0B
bc31c92c4e5d 2 hours ago /bin/sh -c df -h 0B
a57ec209dea1 2 hours ago /bin/sh -c rm -rf /var/lib/postgresql/10/main 0B
d855a6725873 2 hours ago /bin/sh -c df -h 0B
0455466d8e09 2 hours ago /bin/sh -c #(nop) COPY file:e4b17e62219672ee… 20.8GB
So, I can see that during the expansion of the tar file and moving it, the size of the layer is 48.1GB. While the history does not show the layer during the cd & chown, there is the error message that the tar file wasn’t big enough. I assume this means that there is a tar file which is associated with the layer and that it is not or cannot be large enough. I also assume that docker is making the size decision based on the command(s) being executed.
While the cd / chown command is running, I do see an intermediate container being used while running docker stats which looks like:
CONTAINER ID a1c619f7698d
NAME goofy_dhawan
CPU % 93.43%
MEM USAGE / LIMIT 130.2MiB / 11.71GiB
MEM % 1.09%
NET I/O 1.04kB / 0B
BLOCK I/O PIDS 42.8GB / 194GB 2
I can get around this problem by doing:
RUN mv /home/renderaccount/main /var/lib/postgresql/10/main && cd /var/lib/postgresql/10/ && chown -R postgres:postgres main
In this case, it appears all of the commands are executed on a layer which is large enough and everything succeeds.
What I am interested in learning is whether or not this would be considered a docker bug or if this is intended behavior. I am not sure what other information I might be able to provided, but if more is required, please let me know.
This is related to https://forums.docker.com/t/need-to-solve-error-processing-tar-file-no-space-left-on-device/80338/2