How to migrate docker volume from hoste to another host

[azul12]

HI,
I want to migrate a volume from old host to a new host,so I run the container in the new host using docker compose and the app ceates a brand new volumes in /var/lib/docker/volumes/db_mysql/_data but I have already the a backup of the volume in the old host in (/var/lib/docker/volumes/db_mysql/_data) what I did is to delete _data of the new host and replace it with the _data of the old host, but It did not work.
did I missed somthing please ? permissions or something like that, thanks in advanced.

[bluepuma77]

For this reason I prefer to use bind mounts over Docker volumes. Don’t mess manually with the Docker volume files.

You can copy the files from a volume to your host and back, check docker container cp.

[azul12]

thanks for your time, in fact the old server is crushed…I got the volume backup using the rescu mode.i can not use [docker container cp]…

[meyay (Metin Y.)]

Did you delete the old volumes? If not, you can still use an ephemeral container to back up one or more volumes as tar file, copy the tar file(s) over to the new host, and use an ephemeral container to restore them back into a volume:

Migrate Docker Containers

Welcome! Also, It is always good to know how to actualy backup your persisted data, isn’t it? You will want to have a solution that allows to keep a could of backups of each volume for at least a couple of days. Once all containers are stopped, something like this should be sufficient to backup all volumes: for volume in $(docker volume ls -q); do docker run -ti --rm -v $volume:/source -v ./backups:/backups alpine bash -c 'cd /source; tar czvf /backups/$volume.tar.gz' done And this to c…

[azul12]

@meyay thanks for your time ser, well I have retreived the backup as tar.gz manualy from the old machine,because as i explained above i usef the rescue mode. so now the question is how i copy the backup.tar.gz to /var/lib/docker/volumes of the new machine ? if you could help please
thanks in advanced

[meyay (Metin Y.)]

The idea and example scripts are shared in the topic I linked.

[azul12]

i do not know how to projet this script on my backup.tar.gz

for backup in $(ls -1 backups); do
    docker create volume ${backup%%.tar.gz}
    docker run -ti --rm -v ${backup%%.tar.gz}:/target -v ./backups:/backups alpine bash -c 'cd /target; tar xzvf /backups/$backup'
 done

because for me i already delete _data of the new host and replace it with the _data of the old host, but It did not work. maybe it is i missed some chmod or chown on the folder ? what do you think ser ?

[meyay (Metin Y.)]

It is a sign of bad practice to work directly in /var/lib/docker/or its subfolders. I am not going to give advice on anything that relates to modifying anything in the /var/lib/docker folder directly.

It should be something like this:

# name of the docker volume 
VOLUME=yourvolume
# path to the folder where the far file stored, without the actual file name
BACKUP_FOLDER=/path/to/folder/where/the/tar/is
# actuall file name inside the folder above
BACKUP_FILENAME=mybackup.tar.gz

docker create volume ${VOLUME}
docker run -ti --rm -v ${VOLUME}:/target -v  ${BACKUP_FOLDER}:/backups alpine bash -c 'cd /target; tar xzvf /backups/${BACKUP_FILENAME}'

You need to replace the values of the three variables in the beginning with values that reflect your given situation.

[azul12]

aah maybe I got what do you want to say ser.
in fact in my case I’m using a docker compose to run the container …so if i understand it is going to be something like this :

version: '3.1'
services:
  db:
    image: mysql:8.0
   # command: --default-authentication-plugin=mysql_native_password
    environment:
      MYSQL_ROOT_PASSWORD: example
   ports:
      - 33060:3306
    volumes:
      - ${BACKUP_FOLDER}:/var/lib/mysql

volumes:
  ${BACKUP_FOLDER}:

# name of the docker volume 
VOLUME=test
# path to the folder where the far file stored, without the actual file name
BACKUP_FOLDER=/home/leaf/buckup
# actuall file name inside the folder above
BACKUP_FILENAME=backup.tar.gz

correct me if I’m wrong please ?

[meyay (Metin Y.)]

That’s a very surprising post. Why would you add docker compose to the mix?

I am not sure if I am able to help you.

Use the script snippet:

# name of the docker volume 
VOLUME=test
# path to the folder where the far file stored, without the actual file name
BACKUP_FOLDER=/home/leaf/buckup
# actuall file name inside the folder above
BACKUP_FILENAME=backup.tar.gz

docker create volume ${VOLUME}
docker run -ti --rm -v ${VOLUME}:/target -v  ${BACKUP_FOLDER}:/backups alpine bash -c 'cd /target; tar xzvf /backups/${BACKUP_FILENAME}'

BUT: with docker compose your volume name will not be just test, it will be <project name>_test. If you don’t specify the name: <project name> in your compose file, or specify it like docker compose -p <project name> up, the will be folder name where you docker-compose.yml is located.

[azul12]

@meyay thanks for your time and effort.

# name of the docker volume 
VOLUME=debian_db_test
# path to the folder where the far file stored, without the actual file name
BACKUP_FOLDER=/home/leaf/buckup
# actuall file name inside the folder above
BACKUP_FILENAME=backup.tar.gz

UT: with docker compose your volume name will not be just test , it will be <project name>_test
=> yes I agreee with you the real name is debian_db_test

correct me if I’m wrong please, if i run this script :

docker create volume ${VOLUME}
docker run -ti --rm -v ${VOLUME}:/target -v  ${BACKUP_FOLDER}:/backups alpine bash -c 'cd /target; tar xzvf /backups/${BACKUP_FILENAME}'

this will create a volume inside /var/lib/docker/volume called debian_db_test, so now my docker compose should be looke like :

version: '3.1'
services:
  db:
    image: mysql:8.0
   # command: --default-authentication-plugin=mysql_native_password
    environment:
      MYSQL_ROOT_PASSWORD: example
   ports:
      - 33060:3306
    volumes:
      - test:/var/lib/mysql

volumes:
  test:

because when I run docker copose it will create a volume called debian_db_test in this case it will create a new volume or it will erase the prevouis one created by the script ?
thanks for your help

[azul12]

@meyay one more question please :

# name of the docker volume 
VOLUME=test
# path to the folder where the far file stored, without the actual file name
BACKUP_FOLDER=/home/leaf/buckup
# actuall file name inside the folder above
BACKUP_FILENAME=backup.tar.gz

the backup file (backup.tar.gz) when I tar it, it contains something like that : var/lib/docker/volumes/(app_volumes) and among the volumes backup i have something called debian_db_test my question is how i align the script to satisfy my need please, I mean when i run the scrip I should only move the debian_db_test inside the backup.tar.gz (var/lib/docker/volumes/) to /var/lib/docker/volumes of the new host , because I dont know whats gonna happen when i run this script :

docker create volume ${VOLUME}
docker run -ti --rm -v ${VOLUME}:/target -v  ${BACKUP_FOLDER}:/backups alpine bash -c 'cd /target; tar xzvf /backups/${BACKUP_FILENAME}'

thanks a lot for your help

[meyay (Metin Y.)]

@azul12

because I dont know whats gonna happen when i run this script :

Please try, and tinker with it. You can play around and look at the result as much as you want, remove the volume and repeat until the result looks like what you need. Though, the core of your question now is how to use tar. If you want to remove paths, you can append the --strip-components=N (replace N with the subfolder level), and you might also need to append the path you want to restore. It is for you to find out the exact command.

The valuable part of my suggestion is not to provide exact commands you can copy and paste without having to adapt it to your situation yourself. The valuable part is the suggestion to use an ephemeral container to restore the tar.gz into the volume, and an example how this would look like, without messing around with /var/lib/docker folder directly.