When and why is a "full rebuild" required

I’m trying to actually understand this section. I’ve used Docker in the past but it has been 4 years and I can’t recall details. The section says, for example, if I change the Gemfile then I need to do a full rebuild to “sync changes to the Gemfile.lock to the host”. But I don’t understand what is fundamentally different between the Gemfile or any other arbitrary file in the Ruby application and the Gemfile.lock file.

My real actual concern is I am planning on using Docker for a RoR app in development mode so I will be making frequent changes to the files. When and Why will I need to do a full rebuild?

I am not surprised that you were confused by that tutorial. I did not understand it first either. I am not a Ruby developer, so it is normal that I don’t fully understand everything related to Ruby. However, there are some parts of that tutorial that I would not write that way, and I am not talking about “running the image” which is in my point of view is “running the container” as the command also indicates: docker container run ...

It uses an entrypoint

COPY entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/entrypoint.sh
ENTRYPOINT ["entrypoint.sh"]

with this content:

set -e

# Remove a potentially pre-existing server.pid for Rails.
rm -f /myapp/tmp/pids/server.pid

# Then exec the container's main process (what's set as CMD in the Dockerfile).
exec "$@"

but then in the compose file it uses a command which would be the argument of the entrypoint likes this:

entrypoint.sh bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b ''"

which would run this:

set -e

# Remove a potentially pre-existing server.pid for Rails.
rm -f /myapp/tmp/pids/server.pid

# Then exec the container's main process (what's set as CMD in the Dockerfile).
exec bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b ''"

removing the same pid file twice and redefining the command that was defined in the Dockerfile.

It doesn’t really explain what a “full rebuild” is. When I read your questions I immediately thought I would see something like this:

docker compose build --no-cache

otherwise it wouldn’t be a full rebuild using the cache. What it actually means, that the compose file defines a bind mount to mount your local folder into the containers folder /myapp.

      - .:/myapp

The image contains the Gemfile and Gemfile.lock files there, but if you make changes in the Gemfile, you need to regenerate Gemfile.lock (and I will guess because I don’t know Ruby) to build the dependency tree so you can use that file to install your app instead of building the dependency tree every time. If you run

docker compose up --build

It will regenerate the lock file in the new image, but since you mounted the lock file too from your host, you will basically override it in the container with the old lock file. This is why you need to update your local lock file. You could just use the image to run a container and copy the lock file out, but you can also run the build in your running container which would update the lock file that it can see, which is the lock file that you mounted from the host.

So this is how you regenerated the lock file in the image:

WORKDIR /myapp
COPY Gemfile /myapp/Gemfile
COPY Gemfile.lock /myapp/Gemfile.lock
RUN bundle install

And you do the same in the running container:

docker compose run web bundle install

My problem with this solution is this:

  • You run bundle install two times possibly generating two different lock files, but even if they are the same, why would you run a build two times. If it runs quickly, that’s fine, but if it is not, you probably want to avoid that.
  • docker compose run web bundle install would run a new container. It would not execute the bundle install command in a running container, so you would end up with an additional container named like this:
    It can caouse problems if you don’t expect it, but it wil be removed when you run docker compose down, so not a big problem.

So the steps in the tutorial should work, but depending on how fast it works, you could probably do it better by building once and copying the generated content out.

Notr that changing the lock file is not always necessary on thew host. Again, it depends on how Ruby works. If bundle install generates everything that is required to run the application and the generated files (except the lock file) is not in /myapp, you don’ really need to update the lock file on your host, unless you use an IDE which requires it to show you up to date information about your dependencies.

Thank you for your time. The idea that the page I referenced is poorly written never occurred to me. I’ve spent most of today reading and thinking and I’m getting some clarity but I still have some questions. But first, let me bring you up to speed a little about Ruby.

Ruby has “gems” that are package that you add on and it has a utility called “bundler” that that takes a vague list of gems and spews out a precise list of gems that are used in the application. The vague list is called Gemfile and the precise list is called Gemfile.lock. Python has a similar concept.

In development, a user adds what he needs to the Gemfile and then does bundle install and that updates the Gemfile.lock (in the process adding in any dependencies and resolving vague versions such as >1.0 to precise versions such as 1.0.1 . Then in production or deployment, bundle install --deployment looks at the Gemfile.lock and makes sure that those exact gems are used in production.

All this to say that the key here is doing a bundle install. This could be done with the container active by attaching to the container and doing bundle install. This is effectively what docker compose run web bundle install is doing. But here is my confusion.

After that it says to do docker compose up --build. The Postgres container doesn’t need any build and the Ruby container, if it is rebuilt, (I think) would do RUN bundle install from the Dockerfile – would it not? Indeed, all of the commands in the Dockerfile have already been done and the only thing that needs to be repeated is the bundle install. But it looks to me that doing the two commands the page suggests does bundle install twice. Is that right?

This is exactly why I wrote I would not do it this way, but if it runs quickly, then it is an easy solution and there is nothing wrong with it during development. docker compose up --build is required only because `docker compose run will not update the image. If you don’t update the image, you may think everything works, because it worked during development. So you push the built image to a registry containing old configs. Or if the downloaded packages are not in the mounted folders, you config will be up-to-date, but the packages not.

Thank you again. I’m slowly building up the compose and docker files from scratch. I spent the last several hours learning that ARG values get wiped by FROM lines.

In my playing around trying to figure out more precisely how things hang together, I discovered that with docker compose build, the Dockerfile executes, then the mounts as specified in the compose file happen, then the CMD / ENTRYPOINT happens. This is rather interesting to me.

Edit: One thing this implies is the use of bundle install as a RUN command isn’t going to work. It will update the Gemfile.lock within the container but then the external Gemfile.lock is going to be mounted over it (when . is mounted over /myapp).

I’m still experimenting so this might confused you but here is the /myapp directory when viewed via docker compose after a build:

pedz@Peace:s000 docker2-fun % docker compose -f docker/compose.yaml run --no-deps web bash                
root@cb335da43453:/myapp# ls

This is because the mount of /myapp happened and then the CMD I did at build time was touch dog.

This is what /myapp looks like if just the web image is run:

pedz@Peace:s000 docker2-fun % docker run -it --entrypoint bash hatred-web
root@f66046af1d61:/myapp# ls
Gemfile  Gemfile.lock
root@f66046af1d61:/myapp# sum Gemfile.lock 
41607     4

You can see the result of bundle install created a Gemfile.lock file during the building of the image but by the time docker compose completes the whole set up and the CMD / ENTRYPOINT executes, it is not going to be visible.

I feel we are discussing the same thing, so I guess I was not clear. The image is just a template. docker compose build will update the template, but you cans till override it. You could override the command and the entrypoint, but you usually don’t have to., especially when you customize the container using environment variables. So the ENTRYPOINT and CMD together will define the final command running in the container. CMD is the argument of ENTRYPOINT.

docker compose build would build your image, updating the “template”, but it is basically asking Docker to build the image. docker compose up -d will run the containers, but it will also build images if it needs to.

You still need that line to update files in the image.

I’m trying to listen… So, when docker compose build is done, the image is saved before the mount and the CMD / ENTRYPOINT is done?

Yes. ENTRYPOINT and CMD is the actual process that you want to containerize. It must be the last step. docker compose build is just an alternative to “docker build <params>” and “docker compose up” is an alternative to “docker run <params>”, except compose does more and runs the build as a dependency of running the containers.