Performance loss when mounting host volumes

Expected behavior

Based on experiences with a Vagrant managed VMWare Fusion Docker host I expect negligible performance impact when mounting a host’s working copy of a project into a container.

Actual behavior

I experience up to a 75x slowdown in a synthetic benchmark of file reads when mounting a host level copy of project source into a container compared to files native to the container. This falls in line with the performance I see on real world projects.

Information

I’ve provided a simple benchmark below that has provided similar results to real world internal applications. These applications are python backed single page javascript applications that make a decent handful of requests on initial page load, resulting in significant concurrent access to the server.

I’ve tested this benchmark against

  • Docker for Mac beta 18

  • Docker Machine via Docker Toolbox 1.11.2

  • Docker 1.11.2 on a VMware Fusion hosted CoreOS Docker Host. See https://github.com/coreos/coreos-vagrant to reproduce.

  • On Docker for Mac

    • ~62x slowdown when mounting a volume for the sequential benchmark
    • ~30x slowdown on the threaded benchmark.
  • On Docker Machine

    • ~13x slowdown when mounting a volume for the sequential benchmark
    • ~13x slowdown on the threaded benchmark
  • On VMware Fusion

    • ~8x slowdown when mounting a volume for the sequential benchmark
    • ~1.5x slowdown on the threaded benchmark

As you can see the docker native VM solutions (virtual box and hyperkit) are performing significantly worse for mounted volume IO throughput. I’ve found these results to be reasonably inline with with full page loads from a web server with reasonably concurrent access. This, for example, can take a 2-3 second page load up to 60 seconds when mounting a volume in for development work. This is obviously not workable.

Note: I’ve elected to mount the entire project repo, code and test data, as a volume rather than only the data as this is a closer model to the trouble I’m experiencing with real world projects.

I’m happy to help in any way if you need further steps to reliably reproduce the problem. FWIW it seems to be known that osxfs performance is still an issue. This is just my two cents in that it is a complete show stopper for me personally.

Steps to reproduce the behavior

  1. Pull the sample benchmark from https://github.com/bwells/dockerio
  2. Run docker-compose up to output threaded and sequence benchmarks for container only and host volume mounted access.