Docker Export Intermediate Size Multiple Times Container Size

Open Source Projects DockerEngine
1

I’m really perplexed by this behavior.

When I export a docker container, the intermediate size is massive compared to the container size. For example a 20GB container with a base disk size of 64GB using devicemapper can sometimes result in 400GB being streamed via the Docker API.

This is the final progress bar of curl:
100 429G 0 429G 0 0 116M 0 --:--:-- 1:03:11 --:--:-- 24.4M

Something similar happens even if I use the command line. At one point I ran out of space in the middle of an export becausedocker export <CONTAINER ID> -o <DESTINATION FILE>" created a 200GB file even though the container itself is 20GB. The intermediate file size varies from 100GB to 400GB. If it finishes, however, the final size is the same as the container size: 20GB.

Anyone know why this is happening? This happens both over command line and the Docker HTTP API.

Thanks.

2

Is this EVERY export you do or just some of them that exhibit this behavior ???

3

Not all containers. Only containers that have been running for a while and grown in size from all the compiling they’re doing. The original base image size is only 2.6 GB but after fetching the code and compiling it, it grows to about 20GB. When I export those containers, that’s when I encounter this issue.

4

Here’s more info if it helps:

Checking for actual container size:

sudo du -d 2 -h /var/lib/docker/devicemapper | grep `docker inspect -f "{{.Id}}" <CONTAINER NAME>`

27G	/var/lib/docker/devicemapper/mnt/11c703195128caf1e749a5ba4cde22329e50c9bf431e08e8a3ca8de4e03bdcf4
4.0K	/var/lib/docker/devicemapper/mnt/11c703195128caf1e749a5ba4cde22329e50c9bf431e08e8a3ca8de4e03bdcf4-init

Docker command used:

docker export -o test.tar 11c

Repeated runs of ls -la

52119482368 Jul 22 10:19 test.tar
52716548096 Jul 22 10:20 test.tar
76456329216 Jul 22 10:26 test.tar
77507661824 Jul 22 10:26 test.tar
77564661760 Jul 22 10:26 test.tar
89140768768 Jul 22 10:29 test.tar
97361149952 Jul 22 10:32 test.tar
108968873984 Jul 22 10:35 test.tar
152427900928 Jul 22 10:48 test.tar
181685911552 Jul 22 10:58 test.tar

Ran out of space at this point.