# Docker save/load performance **URL:** \ **Category:** General\ **Created:** [April 15, 2016, 2:18pm UTC](https://forums.docker.com/t/docker-save-load-performance/9245 "2016-04-15T14:18:20Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![dmaze](https://avatars.discourse-cdn.com/v4/letter/d/ea5d25/32.png) [@dmaze](https://forums.docker.com/u/dmaze)\ **Post date:** [April 15, 2016, 2:18pm UTC](https://forums.docker.com/t/docker-save-load-performance/9245/1 "2016-04-15T14:18:20Z")
We ship our product exclusively as a collection of Docker images, some of which are rather large. Part of our build process `docker save` all of the images into a tar file, and then our product installer `docker load` that. Even with a fair bit of tuning to avoid duplicated large layers, this winds up being about 10 GB uncompressed and about 7 GB compressed. Both the `docker save` and `docker load` steps tend to be really slow. Is there anything that can be readily done to speed them up? We’ve essentially had this problem forever, but a typical setup calls `docker load` via Docker 1.8 on CentOS 7 with a 100 GB LVM volume as devicemapper Docker storage (on an AWS gp2 SSD EBS volume). Loading that 7 GB .tar.gz generally takes 20-30 minutes depending on the exact system. While the exact hardware setup can be out of our control, hints on better setups are still extremely useful. ---
**Author:** ![programmerq](https://sea3.discourse-cdn.com/docker/user_avatar/forums.docker.com/programmerq/32/208_2.png) [@programmerq](https://forums.docker.com/u/programmerq)\ **Post date:** [April 15, 2016, 3:50pm UTC](https://forums.docker.com/t/docker-save-load-performance/9245/2 "2016-04-15T15:50:19Z")
What do `iotop` and `top` look like on the system while the `docker load` or `docker save` commands are running? This does seem to be a bit slow: ``` >>> (7*1024.)/30/60 3.982222222222222 >>> (10*1024.)/30/60 5.688888888888888 ``` In 30 minutes, it averages reading the compressed data at about 4mb/sec, and writing the uncompressed data at 5.6mb/sec. Is your cpu pegged during this operation? ---
**Author:** ![dmaze](https://avatars.discourse-cdn.com/v4/letter/d/ea5d25/32.png) [@dmaze](https://forums.docker.com/u/dmaze)\ **Post date:** [April 15, 2016, 6:46pm UTC](https://forums.docker.com/t/docker-save-load-performance/9245/3 "2016-04-15T18:46:25Z")
Let’s run the experiment. Amazon m4.2xlarge (8 cores, 32 GB RAM) instance, CentOS 7, Docker 1.9.1. Set up with two gp2 EBS disks, a 32 GB root disk and a 100 GB disk dedicated to Docker devicemapper lvm. Ran `docker-storage-setup` to create the volume. Meanwhile, I’ve built my `docker save` image tarball (11.4 GB uncompressed, 6.2 GB gzipped). 218 layers in total with the largest single layers being 2.7 GB, 1.4 GB, 1.4 GB (again), 1.0 GB, 0.5 GB. While `docker load` is running: - 100% CPU being used by `docker`, about 40-60 MB/s write I/O by `docker-untar`; a `kworker` occasionally spends 100% of its time in iowait according to iotop; then - the system looks mostly idle, with some occasional bursts in iotop of iowait for `kworker` and `docker daemon`, but with a load average above 2.0; then - a `docker-applyLayer` process intermittently appears, when it does, getting 120-240 MB/s write; iotop’s top-line summary says 20-120 MB/s write is happening but it’s not obviously attributable to any process; top says a `docker` process is spending 0-40% CPU A hair under 9 minutes total. (Which is less than I had initially said.) That’s an average of 20 MB/s uncompressed data. That doesn’t “feel” bad to me when I look at it that way. Are there tuning things that would improve it? ---
**Author:** ![carlwain74](https://sea3.discourse-cdn.com/docker/user_avatar/forums.docker.com/carlwain74/32/24248_2.png) [@carlwain74](https://forums.docker.com/u/carlwain74)\ **Post date:** [July 26, 2019, 4:23pm UTC](https://forums.docker.com/t/docker-save-load-performance/9245/4 "2019-07-26T16:23:27Z")
That’s a lot of layers to deal with. Can you try and optimize your Dockerfile to reduce the number of layers? ---
**Author:** ![meyay](https://sea3.discourse-cdn.com/docker/user_avatar/forums.docker.com/meyay/32/34000_2.png) [@meyay](https://forums.docker.com/u/meyay)\ **Post date:** [October 31, 2021, 6:57pm UTC](https://forums.docker.com/t/docker-save-load-performance/9245/5 "2021-10-31T18:57:11Z")
I know this is a rather old thread, but there are in fact ways to speed up the process: 1.) instead of saving each repo:tag individually, use a space seperated list of repo:tag elements as argument. As a result image layer’s will be deduplicated. 2.) use pigz, instead of gzip, to benefit from multithreading, The creation/extraction time can be further optimizied by using the `--fast` argument with pigz at the cost of the archive size. The command might look like this: `docker image save ${list of repo:tags} | pigz --fast > ${target_path_to_archive}.tar.gz` ---
**Author:** ![rimelek](https://sea3.discourse-cdn.com/docker/user_avatar/forums.docker.com/rimelek/32/41602_2.png) [@rimelek](https://forums.docker.com/u/rimelek)\ **Post date:** [October 31, 2021, 7:47pm UTC](https://forums.docker.com/t/docker-save-load-performance/9245/6 "2021-10-31T19:47:02Z")
Keep answering old questions, please. I am here to collect all the information. I have a solution in which I intentionally save a lot of images separately. I think I could benefit from pigz. I didn’t know about it. ---
**Author:** ![meyay](https://sea3.discourse-cdn.com/docker/user_avatar/forums.docker.com/meyay/32/34000_2.png) [@meyay](https://forums.docker.com/u/meyay)\ **Post date:** [October 31, 2021, 7:54pm UTC](https://forums.docker.com/t/docker-save-load-performance/9245/7 "2021-10-31T19:54:33Z")
It comes in handy when you have air-gaped environments and rely on “sneaker-net” or if you have a delivery pipeline that requires artefacts to be pulled thru stages, that do not allow other access than to an “exchange share” from the previous stage. ---
**Author:** ![chambm](https://sea3.discourse-cdn.com/docker/user_avatar/forums.docker.com/chambm/32/32471_2.png) [@chambm](https://forums.docker.com/u/chambm)\ **Post date:** [January 31, 2022, 6:26pm UTC](https://forums.docker.com/t/docker-save-load-performance/9245/8 "2022-01-31T18:26:44Z")
I was using bzip2. Switching to pigz made the save step about 6x faster for just 6% increase in file size. Thanks!