Some setting describing where on my host I can see location of resources used by the application - like Images, Containers. Give me an idea what/where they are using on Disk.
Actual behavior
While there are artifacts here Library/Containers/com.docker.docker - there is no explanation of what is what and quite frankly, would like to understand why Docker.qcow2 is 16Gb - yet I have no Images loaded or containers.
We run Macs with SSD’s and disk space is expensive or non-existent. So, I like things today, I like to be in control.
Information
OS X: version 10.11.4 (build: 15E65)
Docker.app: version v1.10.3-beta5
Running diagnostic tests:
[OK] docker-cli
[OK] Moby booted
[OK] driver.amd64-linux
[OK] vmnetd
[OK] lofs
[OK] osxfs
[OK] db
[OK] slirp
[OK] menubar
[OK] environment
[OK] Docker
[OK] VT-x
Docker logs are being collected into /tmp/20160404-235459.tar.gz.
Your unique id in bugsnag is: 99662941-FBFC-43FD-810E-A2F75EDBF85A
Please quote this in all correspondence.
After creating and removing many images my Docker.qcow2 becomes very large. I found a solution how to shrink it. It creates new image based on the old and reduces unused space. This works for me:
Thanks for the report. Yes we’re aware that disk usage isn’t exactly transparent right now.
We’re working on several improvements in this area to help better manage the diskspace used by the VM. If you find that the image has grown too big and you aren’t worried about any of the images stored you can safely delete the QCOW2 file and restart Docker for Mac. This is only a temporary solution though - watch out for improvements in the changelog soon
Although qemu-img command can shrink the Docker.qcow2 file size, but not help too much. Because after removing docker image files, the unused garbage images still remain in the ext4 file system under Docker.qcow2, and these garbage contents resided in ext4 file system cannot be swept away merely executing qemu-img command.
To clean the garbage, one should tend to fill zero bytes in the free space of ext4 file system, and only then could Docker.qcow2 be reduced by qemu-img command.
So there are several extra steps to reduce Docker.qcow2 should be done under Linux system, briefly discussed below:
screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty to access docker host
login as root
dd if=/dev/zero of=/mnt/docker/zero bs=4096 to fill zero bytes into the free space of the ext4 file system under Docker.qcow4, until the disk is full
rm /mnt/docker/zero then all the free space of Docker.qcow2 would be filled by zero bytes
logout, and detach screen
After above steps manipulated in docker host, Docker.qcow2 can then be reduced by executing: /Applications/Docker.app/Contents/MacOS/qemu-img convert -O qcow2 Docker.qcow2 Docker2.qcow2
BTW, I think it is too cumbersome to do such file reducing processes, and I have an suggestion for implementing storage management in future versions. I suggest that the files (aka aufs images) in /var/lib/docker had better not be placed under Docker.qcow2 disk image, since it is too varying when adding/removing files frequently, and it is too easy to produce non-zero garbage contents in ext4 file system which cannot be swept away by qemu-img command. Instead, these files may be placed under Mac OS X native file system, then establish file accesses in OS X file system when accessing /var/lib/docker in docker host. This simplifies the storage management, and users can direct see how the docker files act, similar to those in Linux system
This Docker.qcow2 file is indeed scary. Mine had 60GB, so I just followed the instructions here and closed Docker, deleted the file and opened Docker again. Its size was stable at 1.1GB after I opened Docker. I went away from computer for a few minutes, Mac OS X locked from inactivity and when I’m back, the file was increasing in size again pretty fast - it’s already at 17GB and I’m still doing absolutely nothing. Is this a bug?
Issue has made Docker unusable. Simply starting the docker app, no containers running, and Docker.qcow2 grows by about 2MB/sec, fills the disk rapidly.
Maybe it wasn’t actually fixed, not sure. I stopped Docker, removed the Docker.qcow2 file (which was 56G before!), restarted Docker (beta13.1 build 8193), and after a couple of minutes the file has grown to 1.1G. “docker images” shows nothing. There’s clearly no reason for a 1GB file when Docker isn’t actually doing anything yet…although it seems to have stopped at 1.1G and isn’t growing any more. It’s better than 56G but still…
The qcow file is a block disk image that grows when unwritten blocks in the 64GB virtual device are written so the sparsity of writes due to ext4 disk layout causes an initially large increase in image size. We are investigating alternatives to the block device but they are not ready yet. In conclusion, while the behavior you observe, @masonck, is not ideal it is working as intended.
Thanks for participating in the Docker for Mac Beta!