Crash and extremely slow ruby app boot time with shared gems folder

Docker Desktop for Mac
1

Expected behavior

Ruby applications sharing a gems folder (via a mount from the host machine) should be fast and not crash.

This used to work fine with a docker-machine based on boot2docker, my volumes were mounted via nfs at the time.

We share gems because we run about 5-6 rails/ruby apps that use a lot of the same gems. It’s to speed up installing gems and to persist the gems in the event of the virtual machine being torned down.

Actual behavior

Starting any ruby app that use a shared volume (mounted from the host) takes minutes to start, made worse the more gems there are.

Running bundle install is fast though, but any operation that needs to load the gems is having issues.

I ran strace on a booting rails app and got tens of thousands (if not a million) of lines like:

[pid   464] 15:50:05.873083 open("/ruby_gems/2.2/gems/rails-4.2.5.1/lib/action_controller/caching/fragments.rb", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

Eventually, after starting a few apps, the vm seemed to have crashed, the logs indicated:

2016-04-29 12:26:09.092 PM Docker[27911]: 
Assertion failed: (sc->reply_cons != sc->reply_prod), function send_response_common, file src/pci_virtio_sock.c, line 763.

Information

pinata diagnose -u                                                              ✭
OS X: version 10.11.4 (build: 15E65)
Docker.app: version v1.11.0-beta9
Running diagnostic tests:
[OK]      docker-cli
[OK]      Moby booted
[OK]      driver.amd64-linux
[OK]      vmnetd
[OK]      osxfs
[OK]      db
[OK]      slirp
[OK]      menubar
[OK]      environment
[OK]      Docker
[OK]      VT-x
Docker logs are being collected into /tmp/20160429-124238.tar.gz
Most specific failure is: No error was detected
Your unique id is: 6FE1F722-8F98-4095-BA99-0B554BC1D31B
Please quote this in all correspondence.

Steps to reproduce the behavior

  • Save this as a Dockerfile in an empty directory:
FROM ruby:2.2

ENV GEM_HOME /ruby_gems/2.2
ENV PATH /ruby_gems/2.2/bin:$PATH

ENV BUNDLE_APP_CONFIG $GEM_HOME
ENV BUNDLE_PATH $GEM_HOME
ENV BUNDLE_BIN $GEM_HOME/bin

RUN mkdir -p /app
WORKDIR /app

CMD 'bash'
  • Save this as a Gemfile in the same directory;
source "https://rubygems.org"
gem "rails"
  • From the same directory, run: docker run -it -vpwd/ruby_gems:/ruby_gems -vpwd`:/app <image_name>
  • Inside the container, run:
    • gem install bundler --no-ri --no-rdoc
    • bundle install
    • bundle exec rails new test
    • cd /test
    • bundle exec rails s
    • Wait for the vm to crash (maybe) or for rails to start.

If you strace the pid of that process from inside xhyve (by using a screen session), you’ll see those not found errors.

2

We have the same issue, on multiple machines.

Just starting a Rails console takes over 20 minutes.

I hope this is just some minor issue that’s causing crazy times like that an not an architectural issue. We’ve been running our own solution for Mac since a while since Virtualbox shared folders and NFS are way too slow for us, and Rsync is also problematic (mainly because it’s one way only). We’ve been using Unison (https://www.cis.upenn.edu/~bcpierce/unison/), which is tricky to setup and has delays when syncing, but then speed is unbeatable. I was hoping that the “native” solution would get at least close to that speed.

3

To extend Michael’s comment: the culprit seems to be the require calls. I tried to run bundle exec irb (which booted up instantly) and require each gem by hand. Each call takes ages (e.g. require 'active_support/all'), but smaller gems do load faster. Something really wrong with path lookup.

After a quick glance at the $: variable (lookup path), I tried requiring gems that appear early in the lookup path and they do seem to load faster than those at the end of the lookup path. However, that might be a coincidence, so YMMV.