File access in mounted volumes extremely slow, CPU bound

I don’t see why host -> docker writing would ever be slow - or at least the writing to the native disk part. I may be missing something though. The slowness with reading is still there sadly so anything that reads a lot of files are still slow.

Hi all, I think I’m hitting on the same problem. Filed an issue https://github.com/docker/docker/issues/25656 and was redirected to https://github.com/docker/docker/issues/21485 which is probably the best technical recap of these types of issues I’ve read so far.

Would love an update on where this is going with the Docker for Mac roadmap. We use a dozen or so containers when developing locally, 2 of them being Gulp watchers. The filesystem scanning simply tears into the host’s CPU when it runs.

Edit: I just spotted @dsheets reply from a few weeks ago. Great response and thanks for the update! I can’t give as much detail as the post requested, but know that the two biggest pain points on my stack have been:

  • gulp 3 watch processes, which are based on the standard Node 5-slim Hub image. These hurt us the most because they sit idly in the background during development but still peg the CPU due to their filesystem monitoring.
  • Validation routines on our code using PHP Code Sniffer. This is an on-demand process which tokenizes entire directories of PHP files before analyzing them.

Created a test case for Gulp.
https://github.com/den-t/docker-for-mac-gulp-test/
Currently hard to test in measurable terms. It just shows prolonged periods of CPU load, after npm install finishes.

@taiidani Could you suggest a watcher that will showcase the problem better than what is there.
PRs welcome. :slight_smile:

I recommend using a package like gulp-watch or gulp-nodemon or similar that takes advantage of inotify file system events rather than constantly polling the file system.

1 Like

I’ve got an example repo using gulp-watch over gulp.watch here for the answer to this question

1 Like

I totally missed the “Opcache” workaround hidden in this reply. I’m also a Drupal developer experiencing crippling slowness when doing drush cc all.

I’ve enabled Opcache on PHP and it seems to speed things up considerably. Probably because it reduces file reads.

On a development machine I have set opcache.revalidate_freq=60, which means the opcache is only valid for 60 seconds. I can’t see the effects of code changes immediately and have to wait for 60s, but I think it is tolerable.

I also spotted that OSXFS process is being in the top of activity monitor. I was writing a typical python + django app where one of the containers was a buil-in debug webserver, which watches for code changes and restarts if necessary.

Turns out that’s the container which drained my battery. After I stopped it osxfs process disappeared from a top.
Is there a way to have a watcher but not kill the CPU? Or maybe Docker is going to do something about that?

I suspect i’m seeing similar performance slowdown with Docker for Mac

Version 1.12.0-a (build: 11213)

I’m seeing differences when running pgbench_tools (https://github.com/gregs1104/pgbench-tools)

Straight postgres container with datavolume ( -v “…/postgresdatadocker:/var/lib/postgresql/data” ) :

set | scale | test | script | clients | workers | limit | tps | avg_latency | 90%< | max_latency | trans
-----±------±-----±-----------±--------±--------±------±-----±------------±------±------------±-------
1 | 1 | 1 | select.sql | 1 | 1 | | 1411 | 0.704 | 0.801 | 35 | 84691
1 | 1 | 2 | select.sql | 2 | 1 | | 2337 | 0.851 | 1.041 | 44 | 140201
1 | 1 | 3 | select.sql | 4 | 1 | | 2603 | 1.42 | 1.958 | 45 | 156191
1 | 1 | 4 | select.sql | 8 | 1 | | 2671 | 2.983 | 3.76 | 44 | 160284
1 | 1 | 5 | select.sql | 16 | 1 | | 0 | | | |
1 | 1 | 6 | select.sql | 32 | 1 | | 0 | | | |
(6 rows)

Versus Virtualbox Ubuntu VM with Docker postgres container and NFS share :

set | scale | test | script | clients | workers | limit | tps | avg_latency | 90%< | max_latency | trans
-----±------±-----±-----------±--------±--------±------±-----±------------±------±------------±-------
1 | 1 | 1 | select.sql | 1 | 1 | | 5259 | 0.187 | 0.227 | 156 | 316306
1 | 1 | 7 | select.sql | 1 | 1 | | 0 | | | |
1 | 1 | 2 | select.sql | 2 | 1 | | 5915 | 0.335 | 0.601 | 156 | 354903
1 | 1 | 3 | select.sql | 4 | 1 | | 7040 | 0.565 | 1.209 | 203 | 422412
1 | 1 | 4 | select.sql | 8 | 1 | | 8791 | 0.906 | 1.352 | 285 | 527490
1 | 1 | 5 | select.sql | 16 | 1 | | 0 | | | |
1 | 1 | 6 | select.sql | 32 | 1 | | 0 | | | |
(7 rows)

I created an issue here: https://github.com/docker/for-mac/issues/77

1 Like

@dsheets I’m wondering if there is any update to this in the past two months?

I am running 1.12.1-beta26.1 and it still seems to have this problem. I am working on a large ruby project and the time it takes to start running tests is measured in minutes, whereas my coworkers working in a VM measures it in seconds. I would love to keep using docker-for-mac, but if reasonable performance is too far out. I may have to switch back to a VM.

Thanks.

1 Like

Was in the same situation (large Ruby app) … use http://docker-sync.io for now.

Yes, I’ve been meaning to try it out. In the meantime I’ve switched to docker-machine and am getting about a 5x speedup. There are some features like user remapping which seem very handy.

What I was more hoping for though is some kind of Idea on the progress of having this solved natively. Perhaps some github issues to watch?

Any update to this issue?

Hi everybody,

same issue here.

Is there already a status update/estimate you could give?
When might this be fixed?

Would also appreciate an update. I had to revert back to using a VirtualBox VM due to volume performance.

+1
Having issues with the slow performance as well.

@dsheets Could somebody from the Docker staff please provide some feedback here.

2 Likes

Earlier in this thread, I suggested making it an option to mount volumes via NFS, and expose a service that could receive file-change notifications out-of-band.

As this has continued to be a problem for my team, I wrote exactly that. FS-EventBridge is a tiny, statically compiled Rust app that opens a TCP server to receive streaming file change notifications. On the host mac, one can run fsbridge to listen for and stream fsevents from any folder into the eventbridge server. We’ve also integrated it into DevLab (our little docker-compose alternative to support a different kind of workflow) so project folders automatically stream their events. For anyone interested in testing this setup, I put together a bootstrapper called DevBox that will configure it all for you. Just uninstall docker-machine first if you already have it installed from somewhere other than brew.

fsbridge is perfectly capable of monitoring large folders (I had it running for my entire user folder at one point – definitely not ideal because it’ll pick up ~/Library and such, but everything still ran fine). So, with a few more lines of code to mitigate any connection issues and exclude unnecessary folders, this could easily be run on startup though a LaunchAgent or otherwise, maybe even pointed to a user-specified projects/workspace folder. That way anyone’s tooling, including docker-compose, would continue working fine without the need to manually run the fsbridge client first.

This setup is screaming fast, giving us near-native performance with two-way volumes and instant inotify events, with any root-created files on the VM’s mounted volume getting the UID and GID of the mac user, all with barely a blip on the CPU usage graph during filesystem-intensive operations, and no additional containers running. It’s the best pieces of all the different solutions discussed here. Please consider this solution for the official app!

4 Likes

Significant problem for me as well. Processes are all disk-access bound, when they should be CPU-bound… I’ll understand in beta, but I’ll defect if this is still there in production