Npm install caching, with optimized build file

Hi - is there any way to cache npm install for changes to package.json - ie. with an optimised Dockerfile, I do the following, but any add/remove from dependencies in package.json causes an entire reinstall?

  COPY package.json .
  RUN npm install
  COPY . .

Note the 2nd copy (COPY . .) is the source, which isnt copied over anywhere in the dockerfile before the COPY package.json.

I could look into using a local npm cache onto the host, but have tried this & not immediately working (times out looking for the cache).

I’ve got an interim solution for my own purposes - I have a root image, and am building other images from it that all share common npm packages (and therefore their dependencies) - I’ve added a package.json to the root image & subsequent npm install - so as long as the child container images use the same source folder (eg. WORKDIR src) then only newer or added npm packages will be downloaded - this typically speeds up my child builds by over 95%.

This is fairly similar to what I’ve done with other interpreted languages.

If I find myself fiddling with packages.json (or requirements.txt, or Gemfile), I’ll try to identify the dependencies that I’m not likely to change for my next several immediate tasks. That might be django, or database libraries, or rails-- That sort of thing. I will then create a separate RUN that will pip install or gem install those identified dependencies, and then my subsequent pip install or npm install or whatever will take less time.

Once I’ve kind of “settled” on what I think dependencies should be for a while, I’ll squash it back down and let my CI system build from there.

1 Like

Would mapping a Volume between the dock and your build machine’s npm cache work?

1 Like

@bamboojim can you map a volume at a point in the build process? What impact does that have on the docker cache? I’d also need to check how to use a folder outside of the package.json location as a cache, while keeping the npm registry as primary …

Sounds sensible - squashing is something I havent looked into yet!