Bootstrapping a Rails Edge App - Bundler Error inside Docker

docsnyder · October 14, 2015, 9:22pm

Hi,
after reading the documentation about creating a Rails 4 App inside an Docker image via docker-compose (-> http://docs.docker.com/compose/rails/) i have tried to get this running with an Rails5 app. (For the sake of simplicity I have omitted the second image with the database.)

I have exchanged the Gemfile in the aforementioned example to install Rails from github. But when I run the command docker-compose run web bundle exec rails new . --force --skip-bundle to create the app i receive following error (full console log at the end of this post):

The git source git://github.com/rails/rails.git is not yet checked out. Please run `bundle install` before trying to start your application

and the rails app is not created. I have tried to identify the source of this error but had to realize that I am neither profound enough in Bundler nor do I understand enough of Docker to fix it.

What I know so far:

although the bundle install command from the Dockerfile seems to be executed no Gemfile.lock is created or visible in the volume
when I open a Shell inside the Docker container, run bundle install and then execute the command to create the rails app everything is alright
- the rails-gem is installed from github, but there are different SHAs in different folders
- /usr/local/bundle/cache/bundler/git/rails-16a5e918a06649ffac24fd5873b875daf66212ad
- /usr/local/bundle/bundler/gems/rails-3ad381c3f859/

I have no idea what could be the cause for this error or why the app is not created, if it is a Bundler misconfiguration, a wrong Dockerfile or my total misunderstanding of Docker/Compose

Thanks for any suggestions how to fix this and get things running

bye
Martin

Here is the Dockerfile

FROM ruby:2.2.3
RUN apt-get update -qq && apt-get install -y build-essential nodejs
RUN mkdir /myapp
WORKDIR /myapp
ADD Gemfile /myapp/
RUN bundle install
ADD . /myapp

the docker-compose.yml

web:
  build: .
  command: bundle exec rails s -p 3000 -b '0.0.0.0'
  volumes:
    - .:/myapp
  ports:
    - "3000:3000"

and last but not least the Gemfile that I have successfully used to bootstrap a Rails5 app outside of Docker

source 'http://rubygems.org'

gem 'rails', :github => 'rails/rails'  
gem 'rack', :github => 'rack/rack'  
gem 'arel', :github => 'rails/arel'  
gem 'sprockets', github: 'rails/sprockets', branch: 'master'  
gem 'sprockets-rails', github: 'rails/sprockets-rails', branch: 'master'

The output when running docker-compose run web bundle exec rails new . --force --skip-bundle

Building web...
Step 0 : FROM ruby:2.2.3
 ---> 5029dc6f1b9e
Step 1 : RUN apt-get update -qq && apt-get install -y build-essential nodejs
 ---> Running in 440ce00c06d6
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  dpkg-dev fakeroot libalgorithm-diff-perl libalgorithm-diff-xs-perl
  libalgorithm-merge-perl libc-ares2 libdpkg-perl libfakeroot
  libfile-fcntllock-perl libtimedate-perl libv8-3.14.5
Suggested packages:
  debian-keyring
The following NEW packages will be installed:
  build-essential dpkg-dev fakeroot libalgorithm-diff-perl
  libalgorithm-diff-xs-perl libalgorithm-merge-perl libc-ares2 libdpkg-perl
  libfakeroot libfile-fcntllock-perl libtimedate-perl libv8-3.14.5 nodejs
0 upgraded, 13 newly installed, 0 to remove and 7 not upgraded.
Need to get 4900 kB of archives.
After this operation, 12.5 MB of additional disk space will be used.
Get:1 http://httpredir.debian.org/debian/ jessie/main libc-ares2 amd64 1.10.0-2 [76.7 kB]
Get:2 http://httpredir.debian.org/debian/ jessie/main build-essential amd64 11.7 [7114 B]
Get:3 http://httpredir.debian.org/debian/ jessie/main libtimedate-perl all 2.3000-2 [42.2     kB]
Get:4 http://httpredir.debian.org/debian/ jessie/main libalgorithm-diff-xs-perl amd64 0.    04-3+b1 [12.2 kB]
Get:5 http://httpredir.debian.org/debian/ jessie/main libfile-fcntllock-perl amd64     0.22-1+b1 [36.4 kB]
Get:6 http://httpredir.debian.org/debian/ jessie/main libdpkg-perl all 1.17.25 [1070 kB]
Get:7 http://httpredir.debian.org/debian/ jessie/main libv8-3.14.5 amd64 3.14.5.8-8.1     [1269 kB]
Get:8 http://httpredir.debian.org/debian/ jessie/main nodejs amd64 0.10.29~dfsg-2 [648 kB]
Get:9 http://httpredir.debian.org/debian/ jessie/main dpkg-dev all 1.17.25 [1544 kB]
Get:10 http://httpredir.debian.org/debian/ jessie/main libfakeroot amd64 1.20.2-1 [44.7 kB]
Get:11 http://httpredir.debian.org/debian/ jessie/main fakeroot amd64 1.20.2-1 [84.7 kB]
Get:12 http://httpredir.debian.org/debian/ jessie/main libalgorithm-diff-perl all 1.19.    02-3 [51.7 kB]
Get:13 http://httpredir.debian.org/debian/ jessie/main libalgorithm-merge-perl all 0.08-2     [13.5 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 4900 kB in 16s (296 kB/s)
Selecting previously unselected package libc-ares2:amd64.
(Reading database ... 21062 files and directories currently installed.)
Preparing to unpack .../libc-ares2_1.10.0-2_amd64.deb ...
Unpacking libc-ares2:amd64 (1.10.0-2) ...
Selecting previously unselected package libtimedate-perl.
Preparing to unpack .../libtimedate-perl_2.3000-2_all.deb ...
Unpacking libtimedate-perl (2.3000-2) ...
Selecting previously unselected package libdpkg-perl.
Preparing to unpack .../libdpkg-perl_1.17.25_all.deb ...
Unpacking libdpkg-perl (1.17.25) ...
Selecting previously unselected package dpkg-dev.
Preparing to unpack .../dpkg-dev_1.17.25_all.deb ...
Unpacking dpkg-dev (1.17.25) ...
Selecting previously unselected package build-essential.
Preparing to unpack .../build-essential_11.7_amd64.deb ...
Unpacking build-essential (11.7) ...
Selecting previously unselected package libfakeroot:amd64.
Preparing to unpack .../libfakeroot_1.20.2-1_amd64.deb ...
Unpacking libfakeroot:amd64 (1.20.2-1) ...
Selecting previously unselected package fakeroot.
Preparing to unpack .../fakeroot_1.20.2-1_amd64.deb ...
Unpacking fakeroot (1.20.2-1) ...
Selecting previously unselected package libalgorithm-diff-perl.
Preparing to unpack .../libalgorithm-diff-perl_1.19.02-3_all.deb ...
Unpacking libalgorithm-diff-perl (1.19.02-3) ...
Selecting previously unselected package libalgorithm-diff-xs-perl.
Preparing to unpack .../libalgorithm-diff-xs-perl_0.04-3+b1_amd64.deb ...
Unpacking libalgorithm-diff-xs-perl (0.04-3+b1) ...
Selecting previously unselected package libalgorithm-merge-perl.
Preparing to unpack .../libalgorithm-merge-perl_0.08-2_all.deb ...
Unpacking libalgorithm-merge-perl (0.08-2) ...
Selecting previously unselected package libfile-fcntllock-perl.
Preparing to unpack .../libfile-fcntllock-perl_0.22-1+b1_amd64.deb ...
Unpacking libfile-fcntllock-perl (0.22-1+b1) ...
Selecting previously unselected package libv8-3.14.5.
Preparing to unpack .../libv8-3.14.5_3.14.5.8-8.1_amd64.deb ...
Unpacking libv8-3.14.5 (3.14.5.8-8.1) ...
Selecting previously unselected package nodejs.
Preparing to unpack .../nodejs_0.10.29~dfsg-2_amd64.deb ...
Unpacking nodejs (0.10.29~dfsg-2) ...
Setting up libc-ares2:amd64 (1.10.0-2) ...
Setting up libtimedate-perl (2.3000-2) ...
Setting up libdpkg-perl (1.17.25) ...
Setting up dpkg-dev (1.17.25) ...
Setting up build-essential (11.7) ...
Setting up libfakeroot:amd64 (1.20.2-1) ...
Setting up fakeroot (1.20.2-1) ...
update-alternatives: using /usr/bin/fakeroot-sysv to provide /usr/bin/fakeroot (fakeroot)     in auto mode
Setting up libalgorithm-diff-perl (1.19.02-3) ...
Setting up libalgorithm-diff-xs-perl (0.04-3+b1) ...
Setting up libalgorithm-merge-perl (0.08-2) ...
Setting up libfile-fcntllock-perl (0.22-1+b1) ...
Setting up libv8-3.14.5 (3.14.5.8-8.1) ...
Setting up nodejs (0.10.29~dfsg-2) ...
update-alternatives: using /usr/bin/nodejs to provide /usr/bin/js (js) in auto mode
Processing triggers for libc-bin (2.19-18+deb8u1) ...
 ---> 6ec98a135220
Removing intermediate container 440ce00c06d6
Step 2 : RUN mkdir /myapp
 ---> Running in 43cb68ef4d7f
 ---> 471e4fc37d93
Removing intermediate container 43cb68ef4d7f
Step 3 : WORKDIR /myapp
 ---> Running in 49bfcd4d14a4
 ---> 2b6b75bc7a6d
Removing intermediate container 49bfcd4d14a4
Step 4 : ADD Gemfile /myapp/
 ---> ee793fe10e80
Removing intermediate container 76471fceddd7
Step 5 : RUN bundle install
 ---> Running in 991580bad369
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching git://github.com/rails/rails.git
Fetching git://github.com/rack/rack.git
Fetching git://github.com/rails/arel.git
Fetching git://github.com/rails/sprockets.git
Fetching git://github.com/rails/sprockets-rails.git
Fetching gem metadata from http://rubygems.org/...........
Fetching version metadata from http://rubygems.org/...
Fetching dependency metadata from http://rubygems.org/..
Resolving dependencies...
Using rake 10.4.2
Installing concurrent-ruby 1.0.0.pre4
Installing i18n 0.7.0
Installing json 1.8.3 with native extensions
Installing method_source 0.8.2
Installing minitest 5.8.1
Installing thread_safe 0.3.5
Installing tzinfo 1.2.2
Using activesupport 5.0.0.alpha from git://github.com/rails/rails.git (at master)
Installing builder 3.2.2
Installing erubis 2.7.0
Installing mini_portile 0.6.2
Installing nokogiri 1.6.6.2 with native extensions
Installing rails-deprecated_sanitizer 1.0.3
Installing rails-dom-testing 1.0.7
Installing loofah 2.0.3
Installing rails-html-sanitizer 1.0.2
Using actionview 5.0.0.alpha from git://github.com/rails/rails.git (at master)
Using rack 2.0.0.alpha from git://github.com/rack/rack.git (at master)
Installing rack-test 0.6.3
Using actionpack 5.0.0.alpha from git://github.com/rails/rails.git (at master)
Installing globalid 0.3.6
Using activejob 5.0.0.alpha from git://github.com/rails/rails.git (at master)
Installing mime-types 2.6.2
Installing mail 2.6.3
Using actionmailer 5.0.0.alpha from git://github.com/rails/rails.git (at master)
Using activemodel 5.0.0.alpha from git://github.com/rails/rails.git (at master)
Using arel 7.0.0.alpha from git://github.com/rails/arel.git (at master)
Using activerecord 5.0.0.alpha from git://github.com/rails/rails.git (at master)
Using bundler 1.10.6
Installing thor 0.19.1
Using railties 5.0.0.alpha from git://github.com/rails/rails.git (at master)
Using sprockets 4.0.0 from git://github.com/rails/sprockets.git (at master)
Using sprockets-rails 3.0.0.beta3 from git://github.com/rails/sprockets-rails.git (at     master)
Using rails 5.0.0.alpha from git://github.com/rails/rails.git (at master)
Bundle complete! 5 Gemfile dependencies, 35 gems now installed.
Bundled gems are installed into /usr/local/bundle.
 ---> 065ea4796f98
Removing intermediate container 991580bad369
Step 6 : ADD . /myapp
 ---> c6b39ff93ec9
Removing intermediate container d5b8d6d21ed0
Successfully built c6b39ff93ec9
The git source git://github.com/rails/rails.git is not yet checked out. Please run `bundle     install` before trying to start your application

careynation · October 26, 2015, 5:13pm

Since you’re starting from a base ruby image, bundle may not be there yet. Try doing RUN gem install bundle inside your dockerfile

docsnyder · October 26, 2015, 9:05pm

Hi,
Bundler is already installed. The desired workflow does indeed work when using a Gemfile with a released Version of Rails - it is taken from http://docs.docker.com/compose/rails/.
My problem is that it is not working when using a Gemfile which references the Rails github repo and I cannot figure out what is causing the different behavior.

joostvanrijn · November 10, 2015, 7:37pm

I’m having the exact same issue, did you by any chance solve this?

docsnyder · November 11, 2015, 6:57am

No but I have not looked into it for some time. Might try again in the next days.

docsnyder · November 18, 2015, 9:13am

I have found the cause of the problem: it’s a combination of Dockers handling of volumes and Bundler git-sources.

When using a git repository as a source in a Gemfile, Bundler will clone that repo and create an entry in the Gemfile.lock to store the info which SHA to use etc. If I understand the phrase

Because RubyGems lacks the ability to handle gems from git, any gems installed from a git repository will not show up in gem list

in the Bundler docs correctly then the existence of the Gemfile.lock is elementary for Bundler to know which gems are already downloaded via git.

So what was happening when executing the command docker-compose run web bundle exec rails new . --force --skip-bundle for the first time?

The Dockerfile will be executed
1. a new Image will be created from the base image
2. the working dir /myapp will be created inside that image
3. the Gemfile will be copied into that directory
4. bundle install will be executed
  1. the git sources will be cloned into /usr/local/bundle
  2. the file Gemfile.lock will be created in /myapp - but only in the container!
5. the other files from the host will be copied into the container under /myapp
6. the resulting container will be saved as a new image

Quick summary: we now have the Gemfile and Gemfile.lock in the /myapp-Directory in the image but not on the host system.

now the docker-compose command will be run
1. the volumes will be mounted
  1. the current directory from the host (without the Gemfile.lock) will be mounted on the container as /myapp
  2. as stated in Docker-Docs the /myapp-directory will be “overwritten”

If the path [target] already exists inside the container’s image, the [source] mount overlays but does not remove the pre-existing content.

so when the actual command bundle exec rails new is executed the Gemfile.lock is not visible inside the container anymore.

I have not found a way to get the Gemfile.lock out of the container when building the image so for now I would just change the docker-compose-command into something like

docker-compose run web /bin/bash -c "bundle install; bundle exec rails new . --edge --force --skip-bundle"

which combines the bundle install and bundle execute command and will run after the volume has been mounted.

georgediaz · December 10, 2016, 7:25pm

Here is an article I wrote that sets up a Rails 5 app using Docker compose that is my goto and I think will help you establish a workflow: https://medium.com/@georgediaz/docker-your-clientside-and-api-apps-c675cc3449c8#.n1s8gn632

stevediaz · September 20, 2023, 3:14pm

I’m having the exact same issue, did you by any chance solve this?

marianomatayoshi89 · October 4, 2023, 3:47pm

I’m facing the same exact issue, the only way I solve it for dev purposes is to:

Use the rubygem version, ie not pointing to github
Update the Gemfile to point to the github version
bundle update GEM_NAME
Do a cp -r /usr/local/bundle/bundler/gems/GEM_NAME-HASH_COMMIT/ /usr/local/bundle/gems/GEM_NAME-GEM_VERSION