I have several base images which depend on each other (base1, base2, base3, base4) and then final images which depend on each of the bases (e.g. final1 depends on base1, final4 depends on base4). The dependencies are always by a specific tag which is created at that moment. Whenever I build, I also push the tag to the relevant repo (which are stored in AWS’s ECR container repository)
base4 has a 8G layer, so I really notice when I need to push this layer.
I have built and pushed a base4 version (a while ago). And now I’ve just built a final4 version. Many of its layers are cached but the 8G layer is not (I can tell because it takes ages to push this layer).
I was trying to work out why this is. But I’m not sure how to do this. I can see that I’ve pulled the relevant base4 image (because docker pull says it’s up to date). I can use docker history on base4 and final4 but I can’t see anything that can tell me the identity (hash?) of the layer.
Is there anything I can do to work out why the large layer is having to be pushed again? (or indeed work out if it is actually identical - it should be, but maybe I’ve made a mistake)
An image consists of a manifest + image layers used in the manifest.
By convention, a registry should indicate that an image layer already exists and shouldn’t be pushed again. In your case this doesn’t seem to happen. This has nothing to do with caching or docker itself… unless of course there is a bug, which then high likely would happen with other registries as well.
When you build your final4 image on your host it will be stored in the host’s local image cache. If you push this image the first time, it will push the manifest + all image layers that don’t exist in the registry. If you push the same image (without deleting and rebuilding the image) later again, it will only push the manifest, but no image layers, as they already exist in the registry.
Now let’s assume you delete the image from the local image cache of your host, and clean the build cache and rebuild the image, just a moment after pushing the image, it will result in a new manifest and new image layers. Which, if pushed, will result in every image layer being uploaded again. An image build is repeatable, but not reproducible because timestamps of files or timestamps embedded into compiled binaries (and even the manifest) will be different.
If you build and push base4, then build final4 based on base4 and push final4, the push should detect that the image layers of base4 already exist and should not push them. If it does, consult the ECR documentation (you could be missing a relevant IAM policy), or consult AWS support.
Is there a way (through CLI) to check the identify of a specific layer (in one image) and compare it to the identity in another layer? (Like comparing the SHAs on two git commits /trees )
Similarly is there a way to see if a specific layer is cached (present) locally / or in a remote repository?