Why is "docker history" always <missing> except for the image-id in the first line?

Docker Engine General
,
1

I remember last year when I used Docker, I could clearly see the image-id of each layer through “docker history”. Now they are all <missing>, and in the past, if I only changed the content of the previous layer, all subsequent layers would be rebuilt. Now I have tested and although I have modified the content of the previous layer, only the modified layer has been rebuilt, and the subsequent layers still use caching, This is a huge optimization compared to the cache in my memory, which is very enjoyable, but I am trying to find updates on this related feature.

Server: Docker Engine - Community
Engine:
Version: 24.0.5
API version: 1.43 (minimum version 1.12)
Go version: go1.20.6
Git commit: a61e2b4
Built: Fri Jul 21 20:35:18 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.22
GitCommit: 8165feabfdfe38c65b599c4993d227328c231fca
runc:
Version: 1.1.8
GitCommit: v1.1.8-0-g82f18fe
docker-init:
Version: 0.19.0
GitCommit: de40ad0

2

Image history basically just shows metadata saved in a json file somehwere in the docker data root. The first line is the layer that you always have on your machine. When you download an image from a registry, you will not have the “build layers” only the filesystem layers so you will not have all the image IDs only the IDs that were created by your docker build command. Recent Docker versions use buildkit by default so even if you build an image locally, you won’t see the image ids. I’m not completely sure right now why, but buildkit optimizes the build process which could also be the cause of why you see layers not rebuilt when buildkit can realize that is not necessary. To be honest I don’t know how that works either.

On the other hand, if you build your image this way:

export DOCKER_BUILDKIT=0

docker build ...

it disables buildkit (it still works, but it will be removed in the future) and you will probably see the old behavior.

update

I realized it is probably because Buildkit doesn’t use Docker to build a new layer so you will not have a docker image id.