How do I change the Docker image installation directory?

[nhazlett (Nick Hazlett)]

You can change Docker’s storage base directory (where container and images go) using the -g option when starting the Docker daemon.

• Ubuntu/Debian: edit your /etc/default/docker file with the -g option: DOCKER_OPTS="-dns 8.8.8.8 -dns 8.8.4.4 -g /mnt"

• Fedora/Centos: edit /etc/sysconfig/docker, and add the -g option in the other_args variable: ex. other_args="-g /var/lib/testdir". If there’s more than one option, make sure you enclose them in " ". After a restart, (service docker restart) Docker should use the new directory.

Using a symlink is another method to change image storage.

Caution - These steps depend on your current /var/lib/docker being an actual directory (not a symlink to another location).

1. Stop docker: service docker stop. Verify no docker process is running ps faux
2. Double check docker really isn’t running. Take a look at the current docker directory: ls /var/lib/docker/
   2b) Make a backup - tar -zcC /var/lib docker > /mnt/pd0/var_lib_docker-backup-$(date +%s).tar.gz
3. Move the /var/lib/docker directory to your new partition: mv /var/lib/docker /mnt/pd0/docker
4. Make a symlink: ln -s /mnt/pd0/docker /var/lib/docker
5. Take a peek at the directory structure to make sure it looks like it did before the mv: ls /var/lib/docker/ (note the trailing slash to resolve the symlink)
6. Start docker back up service docker start
7. restart your containers

[zogness]

I could not rsync or mv or cp /var/lib/docker to /opt. Yes, I had docker daemon turned off and chkconfig docker off before reboot etc. These attempts to move the data never finished running.

Thankfully I did not have any images or containers to move so I was able to change the default dir in /etc/sysconfig/docker and voila, all the docker “run files” were created in /opt/docker. I didn’t even need to create /opt/docker, the init did that for me.

So bottom line is, if you have no images you need to preserve, just add the path to /etc/sysconfig/docker (rhel) or /etc/defaults/docker (debian) and you’re done. You can then nuke /var/lib/docker.

[bitgandtter (Yasmany Cubela Medina)]

Greate entry, i try the hard solution and it works perfectly. Thanks

[hyzhak (Eugene Krevenets)]

I have tried 1st method but it doesn’t work for me. Docker still uses default storage base directory. It seems that if we already used docker it won’t work.

From log:

Feb 13 09:50:03 ns521998.ip-158-69-122.net docker[18575]: time="2016-02-13T09:50:03.195265953-05:00" level=warning msg="devmapper: Base device already exists and has filesystem xfs on it. User specified filesystem  will be ignored."
Feb 13 09:50:03 ns521998.ip-158-69-122.net docker[18575]: time="2016-02-13T09:50:03.197918418-05:00" level=info msg="[graphdriver] using prior storage driver \"devicemapper\""

UPDATE

I have tried to remove existing data but docker recreates them on service start:

Feb 13 12:38:22 ns521998.ip-158-69-122.net docker[29109]: time="2016-02-13T12:38:22.540846940-05:00" level=info msg="devmapper: Creating filesystem xfs on device docker-9:1-783632-base"
Feb 13 12:38:22 ns521998.ip-158-69-122.net docker[29109]: time="2016-02-13T12:38:22.693869639-05:00" level=info msg="devmapper: Successfully created filesystem xfs on device docker-9:1-783632-base"

So what is wrong with option -g?

I’m using Ubuntu 15.10 x86_64 and
Docker version 1.10.1, build 9e83765

/etc/default/docker:

DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4 -g /home/bitcore/docker-data"

folder /home/bitcore/docker-data exists

on other hand 2nd method with symlink works fine.

[benwaller]

I tried this method, but the instructions for Centos are only accurate for version 6.5 and below. Unfortunately I am running 7.0.

I tried to revert everything by moving the /var/lib/docker directory back to its original location, but now when I run docker images or docker ps -a, I have no containers or images. The docker folder is still the same size as it was.

I’ve tried restarting the VM and the docker service, but still have no images or containers. Any help would be greatly appreciated

[chosenbreed]

Thanks. The first method worked fine. The problem is I get this error when downloading new images:

“failed to register layer: Untar re-exec error: exit status 1: output: link /bin/dnsdomainname /bin/domainname: operation not permitted”

Any ideas?

[kitgustavsson]

I tried the second option with a symlink and everything works untill i am to restart my containers.

When i get to step 6:

1. sudo service docker start --> docker start/running, process 1156

2. sudo docker images --> Cannot connect to the Docker daemon. Is the docker daemon running on this host?

3. sudo service docker status --> docker stop/waiting

I dont know what i do wrong here? Could it be that i have some old files i have missed to remove when i reinstalled docker?

[hihell (Jiusi Liu)]

for CentOS 6 a simpler solution is
edit
/etc/syscofig/docker

add "-g /your/storage/dir" to other_args=
will looks like:
other_args="-g /your/storage/dir"

execute:
cp -r /var/lib/docker /your/storage/dir
to copy everything (image/container) to new location

execute:
service docker restart

vola!

[atulyadavtech]

Hi Team,

The solution worked for me, but we are facing strange space utilization issue after performing above steps

Steps followed:-
Size of the directory 856M /var/lib/docker
tar -zcC /var/lib docker > /ganana-ha/var/lib/var_lib_docker-backup-$(date +%s).tar.gz
Size of the tar file 258M /ganana-ha/var/lib/var_lib_docker-backup-1466335482.tar.gz
tar -xvzf /ganana-ha/var/lib/var_lib_docker-backup-1466335482.tar.gz -C /ganana-ha/var/lib/
Backup mv /var/lib/docker /var/lib/docker-backup
ln -sv /ganana-ha/var/lib/docker /var/lib/
Size of the directory after untar 103G /ganana-ha/var/lib/docker

Question
Why there is a size difference between this process.

Thank You
Atul Yadav

[dvohra]

Delete the tar.gz after extracting.

[atulyadavtech]

tar file location is different and size of the tar file is also less.
So from where this 103GB of data came…

[dvohra]

tar.gz could have compressed files which extracts to more than the size of the tar.gz file.

[pacman230]

Folks on CentOS 7, don’t follow this.

[zevarito (Alvaro Gil)]

I would like to make Docker get images from an EFS/AWS volume, there should not be any performance penalty right?

[atifsaddique]

Really helpful.
Thanks

[rsvalerio (Rodrigo Valerio)]

Hello guys

How can i pass the -g option with the new docker for mac?
I want to change the default images folder too.

Thanks in advance.

[jcmanciot (Jean-Christophe Manciot)]

The -g option on Ubuntu 16.04 with docker-engine 1.12.1-0 from< official PPA is completely ignored:

Before docker-engine is even installed in /etc/default/docker:

DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4 -g /media/actionmystique/SAMSUNG-850-Ext4/Docker"
/var/lib/docker does not exist
/media/actionmystique/SAMSUNG-850-Ext4/Docker is empty

After installation without overwriting the /etc/default/docker file:

ll /var/lib/docker
drwx--x--x  10 root root 4096 Aug 28 14:58 ./
drwxr-xr-x 112 root root 4096 Aug 28 14:58 ../
drwx------   5 root root 4096 Aug 28 14:58 aufs/
drwx------   2 root root 4096 Aug 28 14:58 containers/
drwx------   3 root root 4096 Aug 28 14:58 image/
drwxr-x---   3 root root 4096 Aug 28 14:58 network/
drwx------   2 root root 4096 Aug 28 14:58 swarm/
drwx------   2 root root 4096 Aug 28 14:58 tmp/
drwx------   2 root root 4096 Aug 28 14:58 trust/
drwx------   2 root root 4096 Aug 28 14:58 volumes/

 /media/actionmystique/SAMSUNG-850-Ext4/Docker is empty

Using the symlink is the way to go.
Also, the correct commands to control docker service are:

systemctl stop docker
install the symlink as explained by @nhazlett
systemctl start docker
systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2016-08-28 15:12:22 CEST; 7s ago
     Docs: https://docs.docker.com
 Main PID: 7097 (dockerd)
    Tasks: 24
   Memory: 17.5M
      CPU: 429ms
   CGroup: /system.slice/docker.service
           ├─7097 /usr/bin/dockerd -H fd://
           └─7104 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim docker-containerd-shim --metrics-interval=0 --start-timeout 2m --st

Aug 28 15:12:22 samsung-ubuntu dockerd[7097]: time="2016-08-28T15:12:22.331839947+02:00" level=info msg="[graphdriver] using prior storage driver \"aufs\""
Aug 28 15:12:22 samsung-ubuntu dockerd[7097]: time="2016-08-28T15:12:22.341400411+02:00" level=info msg="Graph migration to content-addressability took 0.00 seconds"
Aug 28 15:12:22 samsung-ubuntu dockerd[7097]: time="2016-08-28T15:12:22.341959470+02:00" level=info msg="Loading containers: start."
Aug 28 15:12:22 samsung-ubuntu dockerd[7097]: time="2016-08-28T15:12:22.346940335+02:00" level=info msg="Firewalld running: false"
Aug 28 15:12:22 samsung-ubuntu dockerd[7097]: time="2016-08-28T15:12:22.587848771+02:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.22.0.0/1
Aug 28 15:12:22 samsung-ubuntu dockerd[7097]: time="2016-08-28T15:12:22.719073186+02:00" level=info msg="Loading containers: done."
Aug 28 15:12:22 samsung-ubuntu dockerd[7097]: time="2016-08-28T15:12:22.719141624+02:00" level=info msg="Daemon has completed initialization"
Aug 28 15:12:22 samsung-ubuntu dockerd[7097]: time="2016-08-28T15:12:22.719159156+02:00" level=info msg="Docker daemon" commit=23cf638 graphdriver=aufs version=1.12.1
Aug 28 15:12:22 samsung-ubuntu systemd[1]: Started Docker Application Container Engine.
Aug 28 15:12:22 samsung-ubuntu dockerd[7097]: time="2016-08-28T15:12:22.723875496+02:00" level=info msg="API listen on /var/run/docker.sock"

[bwulff]

Ob Ubuntu 16.04 Server (64bit) I could successfully change the path by editing the

/etc/systemd/system/docker.service.d

like this

[Service]
EnvironmentFile=-/etc/sysconfig/docker
EnvironmentFile=-/etc/sysconfig/docker-storage
EnvironmentFile=-/etc/sysconfig/docker-network
ExecStart=
ExecStart=/usr/bin/docker daemon -H 0.0.0.0:2376 -H unix:///var/run/docker.sock $OPTIONS \
          $DOCKER_STORAGE_OPTIONS \
          $DOCKER_NETWORK_OPTIONS \
          $BLOCK_REGISTRY \
          --tlsverify \
          --tlscacert=/etc/docker/cert/ca.pem \
          --tlscert=/etc/docker/cert/server-cert.pem \
          --tlskey=/etc/docker/cert/server-key.pem \
          -g /data/docker

Added the line with -g (and don’t forget the ’ ’ in the line before).

[liulanghan (Brian Phillips)]

I did both options on a VPS with an sshfs storage vps connected as the image location. I keep getting an error when doing “docker pull ubuntu” -> failed to register layer: Error processing tar file(exit status 1): lchown /bin/dnsdomainname: no such file or directory

Anyone know how to deal with this?

[leogrande]

Check this

I have followed instructions in Runtime directory and storage driver on Ubuntu 16.04.1:

1. mkdir /etc/systemd/system/docker.service.d

2. create some drop-in.conf file in /etc/systemd/system/docker.service.d , e.g. docker-overlay.conf

3. add these lines into drop-in file, replace “/mnt/docker-data” with your new docker data location:
   [Service]
   ExecStart=
   ExecStart=/usr/bin/dockerd --graph="/mnt/docker-data" --storage-driver=overlay

4. sudo systemctl daemon-reload

5. sudo systemctl restart docker