Docker Community Forums

Share and learn in the Docker community.

How to have /var/lib/docker on a 100G drive?

docker

(Tezarin) #1

Hi all,

I created a new VM and added a new partition (100G) so I can allocate it to docker later. It has the following partitions and disks:

Filesystem Size Used Avail Use% Mounted on
/dev/xvda2 30G 3.8G 27G 13% /
devtmpfs 7.8G 0 7.8G 0% /dev
tmpfs 7.7G 0 7.7G 0% /dev/shm
tmpfs 7.7G 8.4M 7.7G 1% /run
tmpfs 7.7G 0 7.7G 0% /sys/fs/cgroup
/dev/xvdg1 30G 33M 30G 1% /home
/dev/xvdb1 30G 552M 30G 2% /var
/dev/xvde1 30G 33M 30G 1% /var/tmp
/dev/xvdc1 30G 33M 30G 1% /var/log
/dev/xvdd1 30G 33M 30G 1% /var/log/audit
tmpfs 1.6G 0 1.6G 0% /run/user/1000

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 30G 0 disk
├─xvda1 202:1 0 1M 0 part
└─xvda2 202:2 0 30G 0 part /
xvdb 202:16 0 30G 0 disk
└─xvdb1 202:17 0 30G 0 part /var
xvdc 202:32 0 30G 0 disk
└─xvdc1 202:33 0 30G 0 part /var/log
xvdd 202:48 0 30G 0 disk
└─xvdd1 202:49 0 30G 0 part /var/log/audit
xvde 202:64 0 30G 0 disk
└─xvde1 202:65 0 30G 0 part /var/tmp
xvdf 202:80 0 100G 0 disk
└─xvdf1 202:81 0 30G 0 part
xvdg 202:96 0 30G 0 disk
└─xvdg1 202:97 0 30G 0 part /home


Upon installing Docker, I noticed docker did not use that xvdf device and I can see the free device as well as the loopback device created by Docker’s defaults:, it added a new 100G in there:

Filesystem Size Used Avail Use% Mounted on
/dev/xvda2 30G 3.8G 27G 13% /
devtmpfs 7.8G 0 7.8G 0% /dev
tmpfs 7.7G 0 7.7G 0% /dev/shm
tmpfs 7.7G 8.4M 7.7G 1% /run
tmpfs 7.7G 0 7.7G 0% /sys/fs/cgroup
/dev/xvdg1 30G 33M 30G 1% /home
/dev/xvdb1 30G 552M 30G 2% /var
/dev/xvde1 30G 33M 30G 1% /var/tmp
/dev/xvdc1 30G 33M 30G 1% /var/log
/dev/xvdd1 30G 33M 30G 1% /var/log/audit
tmpfs 1.6G 0 1.6G 0% /run/user/1000

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 30G 0 disk
├─xvda1 202:1 0 1M 0 part
└─xvda2 202:2 0 30G 0 part /
xvdb 202:16 0 30G 0 disk
└─xvdb1 202:17 0 30G 0 part /var
xvdc 202:32 0 30G 0 disk
└─xvdc1 202:33 0 30G 0 part /var/log
xvdd 202:48 0 30G 0 disk
└─xvdd1 202:49 0 30G 0 part /var/log/audit
xvde 202:64 0 30G 0 disk
└─xvde1 202:65 0 30G 0 part /var/tmp
xvdf 202:80 0 100G 0 disk
└─xvdf1 202:81 0 30G 0 part
xvdg 202:96 0 30G 0 disk
└─xvdg1 202:97 0 30G 0 part /home
loop0 7:0 0 100G 0 loop
└─docker-202:17-100789143-pool 253:0 0 100G 0 dm
loop1 7:1 0 2G 0 loop
└─docker-202:17-100789143-pool 253:0 0 100G 0 dm


I then created another VM and this time I did not create a new 100G disk. But then when I installed Docker on this VM, I noticed the same 100G pool was created:

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 30G 0 disk
├─xvda1 202:1 0 1M 0 part
└─xvda2 202:2 0 30G 0 part /
xvdb 202:16 0 30G 0 disk
└─xvdb1 202:17 0 30G 0 part /var
xvdc 202:32 0 30G 0 disk
└─xvdc1 202:33 0 30G 0 part /var/log
xvdd 202:48 0 30G 0 disk
└─xvdd1 202:49 0 30G 0 part /var/log/audit
xvde 202:64 0 30G 0 disk
└─xvde1 202:65 0 30G 0 part /var/tmp
xvdg 202:96 0 30G 0 disk
└─xvdg1 202:97 0 30G 0 part /home
loop0 7:0 0 100G 0 loop
└─docker-202:17-100789143-pool 253:0 0 100G 0 dm
loop1 7:1 0 2G 0 loop
└─docker-202:17-100789143-pool 253:0 0 100G 0 dm


My question is how can I have a VM that has 100G allocated to docker so /var/lib/docker ends up on the 100G drive?

Thanks in advance


(Sam) #2

stop the docker daemon
move all the /var/lib/docker to a folder on the space you want
create a symbolic link for var/lib/docker to the new folder
start the docker daemon


(Tezarin) #3

Thanks for your reply. How exactly will I do that? I just created a brand new VM, the xvdf device is set to be 100G. But when I installed docker, it didn’t use the xvdf, instead, it created its own disk:

[root@ip-10-17-1-187 centos]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 30G 0 disk
├─xvda1 202:1 0 1M 0 part
└─xvda2 202:2 0 30G 0 part /
xvdb 202:16 0 30G 0 disk
└─xvdb1 202:17 0 30G 0 part /var
xvdc 202:32 0 30G 0 disk
└─xvdc1 202:33 0 30G 0 part /var/log
xvdd 202:48 0 30G 0 disk
└─xvdd1 202:49 0 30G 0 part /var/log/audit
xvde 202:64 0 30G 0 disk
└─xvde1 202:65 0 30G 0 part /var/tmp
xvdf 202:80 0 100G 0 disk
└─xvdf1 202:81 0 30G 0 part
xvdg 202:96 0 30G 0 disk
└─xvdg1 202:97 0 30G 0 part /home
loop0 7:0 0 100G 0 loop
└─docker-202:17-100789143-pool 253:0 0 100G 0 dm
loop1 7:1 0 2G 0 loop
└─docker-202:17-100789143-pool 253:0 0 100G 0 dm

Thanks


(Sam) #4

the commands are

mv = move
ln = link

stop/start docker, read the online doc…

google search can help u find answers to just about every question… its what I do here if I don’t know.


(Tezarin) #5

I am familiar with the basic commands. The process to achieve what I was looking for is more advanced than that.


(Sam) #6

other have done the same thing without issue…

you cannot change WHERE docker look (/var/lib/docker) so you can only symlink it to the new place…


(Tnelis) #7

The -pool volumes are not disks, they’re virtual loopback devices backed by files in /var/lib/docker (/devicemapper?). Anyway, sdetweil already gave you all you needed:

systemctl stop docker
mv /var/lib/docker $xvdf_mountpoint
ln --symbolic $xvdf_mountpoint /var/lib/docker
systemctl start docker

(Note that it isn’t mounted anywhere yet per your lsblk output.)

I have never tried it, but the Docker engine should follow the link fine.

That being said, an even easier way would be to mount the xvdf device on /var/lib/docker directly.

systemctl stop docker
mount /dev/xvdf1 /var/lib/docker
systemctl start docker

(Also note that your only partition on xvdf is 30G, you might want to increase that. In fact, you don’t have to partition a device if you’re using it entirely for one thing – e.g. one filesystem.)

Again I have never done this though I’m quite confident it would work transparently. I do regularly mount /var/lib/docker/volumes on arbitrary devices (for application data).