Docker Community Forums

Share and learn in the Docker community.

Docker COPY/ADD no longer copies differences?

build

(Lordelph) #1

For a long time, I’ve built an app using Docker with an intermediate container e.g. I’d build myapp:base which has all the core files copied:

FROM ubuntu:17.10
ADD app /app

Now, the main container myapp:release might be created simply with this

 FROM myapp:base 
 ADD app /app

A handful of files might be modified in the app directory, and I’d wind up with a small extra layer.

Recently though, this final ADD has resulted in a much larger layer, and I suspect it’s related to a Docker update (I’m using Docker version 18.02.0-ce, build fc4de44, running on Container Linux 1702.1.0)

Has something changed in how Docker figures out whats different? How can I get back to an efficient build which just has a small number of updated files in a layer?

Steps to reproduce:

#make a dummy dir with a 64kb file in it
mkdir -p files
truncate -s 64k files/64k.file

#base container has a copy of the files
cat << EOF > Dockerfile.base
FROM ubuntu:16.04
COPY files/ /root/
EOF

#derived container should just have any updates
cat << EOF > Dockerfile.derived
FROM q899941:base
COPY files/ /root/
EOF

#build them....
docker build --file Dockerfile.base -t  q899941:base .
docker build --file Dockerfile.derived -t  q899941:derived .

#now let's review the layers
docker history q899941:derived

I get this result

IMAGE               CREATED                  CREATED BY                                      SIZE  
4e1eb5168d55        Less than a second ago   /bin/sh -c #(nop) COPY dir:8e20ede288278c71e…   65.5kB
022626ac5cf0        Less than a second ago   /bin/sh -c #(nop) COPY dir:8e20ede288278c71e…   65.5kB
0458a4468cbc        5 weeks ago              /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B    
<missing>           5 weeks ago              /bin/sh -c mkdir -p /run/systemd && echo 'do…   7B    
<missing>           5 weeks ago              /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$…   2.76kB
<missing>           5 weeks ago              /bin/sh -c rm -rf /var/lib/apt/lists/*          0B    
<missing>           5 weeks ago              /bin/sh -c set -xe   && echo '#!/bin/sh' > /…   745B  
<missing>           5 weeks ago              /bin/sh -c #(nop) ADD file:a3344b835ea6fdc56…   112MB

The top layer is 64kb even though no files changed.


(Lordelph) #2

Seems my issue was caused by an update which caused me to switch to using overlay2 storage driver - the behaviour I was used to appears to only work with overlay. See https://github.com/moby/moby/issues/21950#issuecomment-208881571