File access in mounted volumes extremely slow, CPU bound

I can confirm the same results as @mvdstam. Running phpunit on a custom app in Beta 12 was unbearable. In Beta 13, although still slow, is now somewhat manageable. Keep up the good work guys!

1 Like
Version 1.11.1-beta13 (build: 7975)
16dbe555c7dd4304521b21e8286d83fe4864c15c
time dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB) copied, 19.0928 s, 5.4 MB/s

real	0m19.097s
user	0m0.090s
sys	0m1.130s

I can definitely see a huge improvement indeed with the latest beta13 from the metrics there File access in mounted volumes extremely slow, CPU bound

Here is a new benchmark:
With mounted folder from Host:

$ ab -n10 -c10 http://magento.dev/
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking magento.dev (be patient).....done


Server Software:        
Server Hostname:        magento.dev
Server Port:            80

Document Path:          /
Document Length:        0 bytes

Concurrency Level:      10
Time taken for tests:   8.492 seconds
Complete requests:      10
Failed requests:        7
   (Connect: 0, Receive: 0, Length: 7, Exceptions: 0)
Non-2xx responses:      1
Total transferred:      202759 bytes
HTML transferred:       199673 bytes
Requests per second:    1.18 [#/sec] (mean)
Time per request:       8491.525 [ms] (mean)
Time per request:       849.152 [ms] (mean, across all concurrent requests)
Transfer rate:          23.32 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.1      1       1
Processing:     2 4023 3614.3   5173    8490
Waiting:        0 4021 3613.7   5172    8482
Total:          3 4024 3614.3   5174    8491

Percentage of the requests served within a certain time (ms)
  50%   5174
  66%   6435
  75%   7491
  80%   8074
  90%   8491
  95%   8491
  98%   8491
  99%   8491
 100%   8491 (longest request)

With volume only on the VM:

$ ab -n10 -c10 http://magento.dev/
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking magento.dev (be patient).....done


Server Software:        Apache/2.4.20
Server Hostname:        magento.dev
Server Port:            80

Document Path:          /
Document Length:        33229 bytes

Concurrency Level:      10
Time taken for tests:   1.161 seconds
Complete requests:      10
Failed requests:        0
Total transferred:      337120 bytes
HTML transferred:       332290 bytes
Requests per second:    8.61 [#/sec] (mean)
Time per request:       1161.144 [ms] (mean)
Time per request:       116.114 [ms] (mean, across all concurrent requests)
Transfer rate:          283.53 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.3      1       1
Processing:   395  764 278.1    882    1160
Waiting:      395  764 278.1    881    1159
Total:        396  765 278.2    883    1161

Percentage of the requests served within a certain time (ms)
  50%    883
  66%    905
  75%    965
  80%   1075
  90%   1161
  95%   1161
  98%   1161
  99%   1161
 100%   1161 (longest request)

So I can confirm the general feeling, the beta13 makes it usable now and the change are on the right path we are just not as the near OS speed yet, though.

Beta13 is much more performant on my very simple Symfony example (a default Symfony install) too. Running this:

ab -n 100 -r 127.0.0.1:8080

takes 26.9 seconds instead of 210 seconds! (Iā€™m using a late-2013 Macbook Pro)

I published more detailed results here: https://github.com/michaelperrin/docker-symfony-test/blob/master/README.md#results

Itā€™s still quite slow but itā€™s already a huge step forward.

The http call (read) is maybe fast but write speed of the mounted volumes are still slow.

Tried same test again with beta13, itā€™s faster, but still slow compared to VirtualBox:

  • Macbook Pro - OS X El Capitan v10.11.5

  • Docker app version - Version 1.11.1-beta12 (build: 7528)

Docker Mac Beta:

/Users/andres1 #  time dd if=/dev/zero of=test.dat bs=1024 count=10000
10000+0 records in
10000+0 records out
real	0m 2.40s
user	0m 0.03s
sys	0m 0.15s

Default docker-machine, Oracle VM, mounted /Users using NFS:

/Users/andres1 # time dd if=/dev/zero of=test.dat bs=1024 count=10000
10000+0 records in
10000+0 records out
real	0m 0.07s
user	0m 0.00s
sys	0m 0.03s

~1s improvement compared to beta12

(I donĀ“t know if that really belongs here)

We have major issues with the latest version.

We are using NodeJS/Gulp aund gulp-watch to watch the filesystem for changes.

Up until yesterday it was just very slow but since 1.11.1-beta13 (build: 7975) the watch mechanism is completely broken. It does not recognize any file changes in a long time and when it does the docker container breaks itself completely.

gulp-watch uses internally chokidar which uses ā€œa native extension exposing the Darwin
FSEvents APIā€ (https://github.com/paulmillr/chokidar).

Anyone else experiencing similar issues?

@alexlawrence

Using the search, I was able to find topics more related to the gulp-watch problem. One of them is this one:

HTH,
Alexandre

Macbook Pro ā€“ OS X El Capitan v10.11.5 (15F34)
Docker Mac ā€“ Version 1.11.1-beta13.1 (build: 8193)

# time dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
real	0m 26.45s
user	0m 0.18s
sys	0m 1.19s

How to enable sharing with NFS?
Currently docker for mac is not useable :frowning:

OSX El Capitan 10.11.5 (15F34)
Docker Version 1.11.1-beta13.1 (build: 8193)

The speed extremely slow!
/app is mounted volume.

root@bd98b360b79d:/app# time dd if=/dev/zero of=test bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB) copied, 40.905 s, 2.5 MB/s

real    0m40.936s
user    0m0.260s
sys    0m3.130s

root@bd98b360b79d:/tmp# time dd if=/dev/zero of=test bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB) copied, 1.14822 s, 89.2 MB/s

real    0m1.244s
user    0m0.050s
sys    0m0.440s

I found this article very helpful. TL;DR: You can mount a volume (usually via busybox) that provides speedy access to cached directories. It wonā€™t solve every use case, but for things like gem dependencies itā€™s awesome.

My tests on:
OSX El Capitan 10.11.4
Version 1.11.1-beta13.1 (build: 8193):

root@2d660f476351:/usr/share/nginx/html# time dd if=/dev/zero of=test bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB) copied, 24.8516 s, 4.1 MB/s

real 0m24.856s
user 0m0.170s
sys 0m0.750s

1 Like

I just replied in another thread about a possible (nasty) workaround I found and have set up. This temporary solution works very well for me, although I do not expect to everyone like it. It simply adds another container for syncing changes from the mount to the application folder.

https://forums.docker.com/t/how-to-speed-up-shared-folders/9322/15?u=sveneisenschmidt

I am very optimistic about the Docker people fixing this problem and I can live with a sync container for some time.

Docker for Mac: version: v1.11.1-beta14.5.m
OS X: version 10.11.1 (build: 15B42)
logs: /tmp/20160607-093828.tar.gz
failure: No error was detected

ā€” local

root@9e5d52471dc9:/tmp# dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB) copied, 0.49442 s, 207 MB/s

ā€” mounted

root@9e5d52471dc9:/mounted/folder# dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB) copied, 39.3927 s, 2.6 MB/s

Hey,

Do you guys think this is possible to provide nfs as the alternative filesystem? docker for mac beta is awesome and the only problem is the performance of mounted volumes. If we could use NFS instead, even temporarily, that would make docker for mac super useful!

3 Likes

Yes, ATM I try each docker beta release and still fallback to a VirtualBOX with NFS share of my source code as volume sharing is the main issue of docker beta, other ones are ok to live with.

NFS and friends, donā€™t support fs events (https://developer.apple.com/library/mac/documentation/Darwin/Reference/FSEvents_Ref/index.html) propagation (, thatā€™s why they are building the new osxfs fs layer, using NFS with Docker for mac does not have much sense now, i hope that the performance issues will be mainly solved in the new few weeks, but i feel that the things are much more complicated than expected, just consider that filesystem like NFS are in production since 30+ years and they still suffer problems.

Yes, this is truth. NFS has its flaws. Nevertheless, it is already used by most of us.

As you said NFS is developed since 30+ years - for me, it just proves that it will be super difficult to find a better solution (i still keep fingers crossed).

Still very slow in release candidate. Is this something that has priority for a coming release? The slowness of bound filesystems makes development quite a difficult proposition with certain kinds of projects.


root@ec530c9c1dbd:/# cd /tmp/
root@ec530c9c1dbd:/tmp# dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB, 98 MiB) copied, 0.666999 s, 154 MB/s
root@ec530c9c1dbd:/tmp# cd /project/
root@ec530c9c1dbd:/project# dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB, 98 MiB) copied, 24.3231 s, 4.2 MB/s

I agree. Itā€™s still completely unusable for developing my rails apps. I canā€™t even do real beta testing if rendering a page takes ~30 seconds.