We currently use this to compress the docker save output:
docker save | xz -T 0 > image.tar.xz
With 29.5.3 the image.tar.xz size is 1.7 G, but with 29.6.0 the size is now 2.3 G, an increase of over .5 G - a very significant change in size!
It looks like 29.6.0 now compresses all of what were previously uncompressed tar file at a per file level.
That change seems to prevent better compression from happening at a higher level - that is, compressing across compressed files is not able to improve compression much as compared to compressing a bunch of uncompressed files.
Is there a way to get the behavior we had with 29.5.3 (and earlier), such that we can run compression across the uncompressed files?
Right now the only way I can think of to do this is to run the docker save, untar the result, uncompress compressed files and than re-tar and compress the result.
With the images I tested, I could not reproduce any difference between 29.6.1 and 29.5.3. I can’t see any relevant change in the release notes either.
If I test a much odler version, the exported layers are indeed different and the exported size is different. Some files are just json files some are targz files indeed. Not sure,because I never tested, but the change could also be caused by the new container image store. If that is the case, I guess if you don’t use that image store, that could help, but you can’t just disable it. You would need to pull all images again and optionally remove the images first from the container image store.
I was comparing our build server on Ubuntu 22.04 with docker 29.6.0 against a local “docker save” using 29.5.3 as I can’t see what our previous build server agent had - it was 28.x something - let alone easily run the older version or get a shell on the build server (VM that gets spun up).
I just tried again now, and can see the images are the same size on my local system when I try 29.5.3 vs 29.6.1 (went to 29.6.1 as I just updated it again).
But the image sizes are larger on our build system. So I can focus on what else is different on the build server.
Can you elaborate on what you mean by “the change could also be caused by the new container image store”?
If ypu chekck the shared link, the documentation explains what the containerd image store is. In short, the images are stored by containerd, not Docker directly there could be differences in the format.Since the image store can be changed, the problem could be unrelated to the version itself if the image store was different on the different machines. Containerd can also be configured separately, so while Docker can be configured to use containerd for storing images,
It is not about docker save, but you can find the Exporters documentation of docker buildx