Docker Community Forums

Share and learn in the Docker community.

Copying files is unusably slow in the latest update

(Cyph, Inc.) #1

Expected behavior

The following code should take about 65 seconds to execute:

for project in ; do
	cp -rf shared/* $project/

Actual behavior

It takes about 35 minutes, with my CPU spiked to 185% the whole time. At one point docker ps -a started to hang indefinitely as well, but a reboot fixed that.


OS X: version 10.11.4 (build: 15E65) version v1.11.1-beta10
Running diagnostic tests:
[OK]      docker-cli
[OK]      Moby booted
[OK]      driver.amd64-linux
[OK]      vmnetd
[OK]      osxfs
[OK]      db
[OK]      slirp
[OK]      menubar
[OK]      environment
[OK]      Docker
[OK]      VT-x
Docker logs are being collected into /tmp/20160504-140228.tar.gz
Most specific failure is: No error was detected
Your unique id is: A140216E-8BE9-4C20-8A4F-BC6FE8CFF717
Please quote this in all correspondence.

Steps to reproduce the behavior

  1. Try to copy files inside a Docker container.

(Dave Tucker) #2

Thanks for the report. Can you give us some more information about the type of files you are copying… How many files? Are they mostly small files? How deep are the directory trees? Are they in your $HOME or nested further in a tree of directories etc…

When you say it should take 65 seconds, what are you comparing to?

(Cyph, Inc.) #3

Sure, and thanks for the quick response!

It’s 341 MB of 11,940 files, with the largest file being 5.35 MB, so I’d say a large number of small files.

As far as depth, the most deeply nested directories are 12 levels deep into the shared folder.

The full path of shared inside the Docker container is /cyph/.build/shared, so not under $HOME but not super deeply nested either. Outside of Docker, in my case, the path is $HOME/cyph/master.

As far as the reference time of 65 seconds, I was comparing to running that command outside of a Docker context, although it’s certainly understandable that osxfs would add some overhead on top of that number.

(Cyph, Inc.) #4

Got an update. To work around this, I tried including some code in a different script that would remove unnecessary files from our dependency folders after updating the dependencies and before committing them. To do this, I had a small number of find commands pick out several thousand such files and pass their paths to rm.

The result was that Docker completely froze up apparently indefinitely, docker ps would hang, and my host system’s hostname was inexplicable changed to new-host-N, with N being some number between 3 and 6.

This was consistently reproducible, and it didn’t seem to matter whether I batched the file removal into a single rm operation or executed a bunch of small operations via find -exec rm {} \;.

(Dave Tucker) #5

Thanks for the update! I’ve opened issue (#3095) on our internal tracker for this and one of our engineers will look in to this soon! Thanks for using Docker for Mac!

(Dave Tucker) #6

Good News! This has been fixed and should be available in the next release - Beta 13

(Bouellette) #7

Hi Dave,
Will this fix the general performance issues on shared folders, or is your fix just about his instance hanging?