Question about backing up docker and restoring tests?

farzadha2 · July 16, 2022, 9:07pm

Hi
Currently i have a question about docker backup and restoring
im currently using wordpress site in docker to backup,

version: '3.9'

services:
  # Database
  db:
    image: mysql:8.0

    volumes:
      - /wordpress/db_data:/var/lib/mysql
    labels:
      # This means the container will be stopped during backup to ensure
      # backup integrity. You can omit this label if stopping during backup
      # not required.
      - docker-volume-backup.stop-during-backup=true
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: thepassword
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: thepassword
    networks:
      - wpnet
  # phpmyadmin
  phpmyadmin:
    depends_on:
      - db
    image: phpmyadmin/phpmyadmin
    restart: always
    ports:
      - '8023:80'
    environment:
      PMA_HOST: db
      MYSQL_ROOT_PASSWORD: thepassword
    networks:
      - wpnet
  # Wordpress
  wordpress:
    depends_on:
      - db
    image: wordpress:latest
    ports:
      - '8024:80'
    restart: always
    volumes:
      - /wordpress/html:/var/www/html
    labels:
      # This means the container will be stopped during backup to ensure
      # backup integrity. You can omit this label if stopping during backup
      # not required.
      - docker-volume-backup.stop-during-backup=true
    environment:
      WORDPRESS_DB_HOST: db:3306
      WORDPRESS_DB_USER: wordpress
      WORDPRESS_DB_PASSWORD: thepassword
    networks:
      - wpnet


  backup:
    image: offen/docker-volume-backup:v2
    environment:
      BACKUP_FILENAME: backup-%Y-%m-%dT%H-%M-%S.tar.gz
      BACKUP_LATEST_SYMLINK: backup-latest.tar.gz
    volumes:
      - data:/backup/my-app-backup:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ${HOME}/backups:/archive

networks:
  wpnet:
volumes:
  data:
  db_data:
  html:

i then ran the backup manually

 docker exec wp_backup_1 backup
time="2022-07-14T04:25:34Z" level=info msg="Stopping 2 container(s) labeled `docker-volume-backup.stop-during-backup=true` out of 12 running container(s)."
time="2022-07-14T04:25:34Z" level=info msg="Created backup of `/backup` at `/tmp/backup-2022-07-14T04-25-34.tar.gz`."
time="2022-07-14T04-25-34Z" level=info msg="Restarted 2 container(s) and the matching service(s)."
time="2022-07-14T04-25-34Z" level=info msg="Stored copy of backup `/tmp/backup-2022-07-14T04-25-34.tar.gz` in local archive `/archive`."
time="2022-07-14T04-25-34Z" level=info msg="Created/Updated symlink `backup-latest.tar.gz` for latest backup."
time="2022-07-14T04-25-34Z" level=info msg="Removed tar file `/tmp/backup-2022-07-14T04-25-34.tar.gz`."
time="2022-07-14T04-25-34Z" level=info msg="Finished running backup tasks."

the backups were created in /backups

i deleted my wordpress container and now i need to restore it which this part did not understand correctly

would i

start again the docker compose
stop the containers
restore using the offen/docker container, which that part im confused how to do it?

Thank you

meyay · July 16, 2022, 11:16pm

Does it realy create backups of the service db and wordpresseven though no volume is used? Actualy with the whole compose file you declare 3 volumes, but only the backup container uses the datavolume. Everything else is a bind, which binds a host folder into a container folder - these are not volumes in the sense of volumes.

I must admit that I find it also confusing with your compose file as it is, as wordress does not use a volume at all.

Lets assume for a minute that the wordpress service uses the named volume html instead of the bind from the host folder /wordpress/html, then you would need to run something like this command to restore the data back into the volume:

docker run --rm -v projectname_html:/var/www/html -v /location/of/the/backup/:backup alpine tar xzvf /backup/my-wonderfull.tar  -C /var/www/html

Make sure to replace projectname with the projectname of your compose deplyoment (which defaults to the directory name if not specified), /location/of/the/backup/ with the path to the folder where your backups are located and my-wonderfull-backup.tar.gz with the real name of the archive.

What this command does: it creates a temporary alpine container with the volume mounted in /var/www/html and the backup folder mounted in /backups, then untars the archive into /var/www/html.

I hope it helps. offen/docker-volume-backup looks like a great finding, btw.

Update: I forget to adress this:

Since you used a bind, the data does still exist. The same should be true when deleting the wordpress container, unless you realy used a named volume and deleted it specifiy or by using docker-compose down -v

farzadha2 · July 20, 2022, 4:28pm

thank you so much it helped me alot with that explication

meyay · July 20, 2022, 5:47pm

Please share your final configution and thoughts, as it will help others that stumble accross your post to find the solution you applied as well.

farzadha2 · July 20, 2022, 8:19pm

allright so after the explication you gave me i realized that its better to manage volume mounts, but i needed the bind mounts because i store then in different disks (hence proxmox that i use)

so in that case i was reading and found that i needed persistent volumes with bind mounts which came across this

curl -fsSL https://raw.githubusercontent.com/CWSpear/local-persist/master/scripts/install.sh | sudo bash

after that i created the docker compose something like this and created the volume perfectly

version: '3.9'

services:
  # Database
  db:
    image: mysql:8.0

    volumes:
      - db_data:/var/lib/mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: thepassword
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: thepassword
    networks:
      - wpnet
  # phpmyadmin
  phpmyadmin:
    depends_on:
      - db
    image: phpmyadmin/phpmyadmin
    restart: always
    ports:
      - '8023:80'
    environment:
      PMA_HOST: db
      MYSQL_ROOT_PASSWORD: thepassword
    networks:
      - wpnet
  # Wordpress
  wordpress:
    depends_on:
      - db
    image: wordpress:latest
    ports:
      - '8024:80'
    restart: always
    volumes:
      - html:/var/www/html
    environment:
      WORDPRESS_DB_HOST: db:3306
      WORDPRESS_DB_USER: wordpress
      WORDPRESS_DB_PASSWORD: thepassword
    networks:
      - wpnet

volumes:

   db_data:
    driver: local-persist
    driver_opts:
      mountpoint: /wordpress/db_data
   html:
    driver: local-persist
    driver_opts:
      mountpoint: /wordpress/html

i was also able to backup manually

tar -czvf wordpress.tar.gz /wordpress

and to restore

tar -xvf wordpress.tar.gz

then re run the docker compose file and worked perfectly

But using the offen docker cant seem to get it working i keep getting

failed to deploy a stack: In file '/data/compose/34/docker-compose.yml', volume must be a mapping, not an array. : exit status 1

Which i edited the docker compose file

version: '3.9'

services:
  # Database
  db:
    image: mysql:8.0

    volumes:
      - db_data:/var/lib/mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: thepassword
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: thepassword
    networks:
      - wpnet
  # phpmyadmin
  phpmyadmin:
    depends_on:
      - db
    image: phpmyadmin/phpmyadmin
    restart: always
    ports:
      - '8023:80'
    environment:
      PMA_HOST: db
      MYSQL_ROOT_PASSWORD: thepassword
    networks:
      - wpnet
  # Wordpress
  wordpress:
    depends_on:
      - db
    image: wordpress:latest
    ports:
      - '8024:80'
    restart: always
    volumes:
      - html:/var/www/html
    environment:
      WORDPRESS_DB_HOST: db:3306
      WORDPRESS_DB_USER: wordpress
      WORDPRESS_DB_PASSWORD: thepassword
    networks:
      - wpnet

backup:
    # In production, it is advised to lock your image tag to a proper
    # release version instead of using `latest`.
    # Check https://github.com/offen/docker-volume-backup/releases
    # for a list of available releases.
    image: offen/docker-volume-backup:latest
    restart: always
    environment:
      BACKUP_FILENAME: backup-%Y-%m-%dT%H-%M-%S.tar.gz
      BACKUP_LATEST_SYMLINK: backup-latest.tar.gz
   
volumes:
      - db_data:/backup/my-app-backup:ro
      - html:/backup/my-app2-backup:ro
      # Mounting the Docker socket allows the script to stop and restart
      # the container during backup. You can omit this if you don't want
      # to stop the container. In case you need to proxy the socket, you can
      # also provide a location by setting `DOCKER_HOST` in the container
      - /var/run/docker.sock:/var/run/docker.sock:ro
      # If you mount a local directory or volume to `/archive` a local
      # copy of the backup will be stored there. You can override the
      # location inside of the container by setting `BACKUP_ARCHIVE`.
      # You can omit this if you do not want to keep local backups.
      - /root:/archive      
   
db_data:
    driver: local-persist
    driver_opts:
      mountpoint: /wordpress/db_data
html:
    driver: local-persist
    driver_opts:
      mountpoint: /wordpress/html

i was reading a bit but couldn’t find what could be the issue

Thank you

farzadha2 · July 20, 2022, 8:21pm

i just replied but it seem that it has to be reviewed my post from a mod

meyay · July 20, 2022, 8:36pm

I saw it by mail:

Instead of

backup:
    # In production, it is advised to lock your image tag to a proper
    # release version instead of using `latest`.
    # Check https://github.com/offen/docker-volume-backup/releases
    # for a list of available releases.
    image: offen/docker-volume-backup:latest
    restart: always
    environment:
      BACKUP_FILENAME: backup-%Y-%m-%dT%H-%M-%S.tar.gz
      BACKUP_LATEST_SYMLINK: backup-latest.tar.gz
   
volumes:
      - db_data:/backup/my-app-backup:ro
      - html:/backup/my-app2-backup:ro
      # Mounting the Docker socket allows the script to stop and restart
      # the container during backup. You can omit this if you don't want
      # to stop the container. In case you need to proxy the socket, you can
      # also provide a location by setting `DOCKER_HOST` in the container
      - /var/run/docker.sock:/var/run/docker.sock:ro
      # If you mount a local directory or volume to `/archive` a local
      # copy of the backup will be stored there. You can override the
      # location inside of the container by setting `BACKUP_ARCHIVE`.
      # You can omit this if you do not want to keep local backups.
      - /root:/archive      
   
db_data:
    driver: local-persist
    driver_opts:
      mountpoint: /wordpress/db_data
html:
    driver: local-persist
    driver_opts:
      mountpoint: /wordpress/html

It high likely should be:

backup:
  # In production, it is advised to lock your image tag to a proper
  # release version instead of using `latest`.
  # Check https://github.com/offen/docker-volume-backup/releases
  # for a list of available releases.
  image: offen/docker-volume-backup:latest
  restart: always
  environment:
    BACKUP_FILENAME: backup-%Y-%m-%dT%H-%M-%S.tar.gz
    BACKUP_LATEST_SYMLINK: backup-latest.tar.gz
  volumes:
    - db_data:/backup/my-app-backup:ro
    - html:/backup/my-app2-backup:ro
    # Mounting the Docker socket allows the script to stop and restart
    # the container during backup. You can omit this if you don't want
    # to stop the container. In case you need to proxy the socket, you can
    # also provide a location by setting `DOCKER_HOST` in the container
    - /var/run/docker.sock:/var/run/docker.sock:ro
    # If you mount a local directory or volume to `/archive` a local
    # copy of the backup will be stored there. You can override the
    # location inside of the container by setting `BACKUP_ARCHIVE`.
    # You can omit this if you do not want to keep local backups.
    - /root:/archive      

volumes:
  db_data:
    driver: local-persist
    driver_opts:
      mountpoint: /wordpress/db_data
  html:
    driver: local-persist
    driver_opts:
      mountpoint: /wordpress/html

Note: there is no need to install a volume plugin for this. The local driver is perfectly capable do create named volumes backed by a bind (the html volume is used as example):

volumes:
  html:
    driver_opts:
      type: none
      o: bind
      device: /wordpress/db_data

farzadha2 · July 21, 2022, 4:58am

Thank you so much for the reply, so i added the part that you edited to my docker compose but im getting this error

failed to deploy a stack: The Compose file '/data/compose/43/docker-compose.yml' is invalid because: Invalid top-level property "backup". Valid top-level sections for this Compose file are: version, services, networks, volumes, secrets, configs, and extensions starting with "x-". You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the serviceskey, or omit theversion key and place your service definitions at the root of the file to use version 1. For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/ : exit status 1

This is the docker compose

version: '3.9'

services:
  # Database
  db:
    image: mysql:8.0

    volumes:
      - db_data:/var/lib/mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: thepassword
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: thepassword
    networks:
      - wpnet
  # phpmyadmin
  phpmyadmin:
    depends_on:
      - db
    image: phpmyadmin/phpmyadmin
    restart: always
    ports:
      - '8023:80'
    environment:
      PMA_HOST: db
      MYSQL_ROOT_PASSWORD: thepassword
    networks:
      - wpnet
  # Wordpress
  wordpress:
    depends_on:
      - db
    image: wordpress:latest
    ports:
      - '8024:80'
    restart: always
    volumes:
      - html:/var/www/html
    environment:
      WORDPRESS_DB_HOST: db:3306
      WORDPRESS_DB_USER: wordpress
      WORDPRESS_DB_PASSWORD: thepassword
    networks:
      - wpnet

backup:
  # In production, it is advised to lock your image tag to a proper
  # release version instead of using `latest`.
  # Check https://github.com/offen/docker-volume-backup/releases
  # for a list of available releases.
  image: offen/docker-volume-backup:latest
  restart: always
  environment:
    BACKUP_FILENAME: backup-%Y-%m-%dT%H-%M-%S.tar.gz
    BACKUP_LATEST_SYMLINK: backup-latest.tar.gz
  volumes:
    - db_data:/backup/my-app-backup:ro
    - html:/backup/my-app2-backup:ro
    # Mounting the Docker socket allows the script to stop and restart
    # the container during backup. You can omit this if you don't want
    # to stop the container. In case you need to proxy the socket, you can
    # also provide a location by setting `DOCKER_HOST` in the container
    - /var/run/docker.sock:/var/run/docker.sock:ro
    # If you mount a local directory or volume to `/archive` a local
    # copy of the backup will be stored there. You can override the
    # location inside of the container by setting `BACKUP_ARCHIVE`.
    # You can omit this if you do not want to keep local backups.
    - /root:/archive      

volumes:
  db_data:
    driver: local-persist
    driver_opts:
      mountpoint: /wordpress/db_data
  html:
    driver: local-persist
    driver_opts:
      mountpoint: /wordpress/html

meyay · July 21, 2022, 9:27am

Please make yourself equainted on how the structure of a docker-compose file is. Keep in mind that indentation has semantics in a yaml file and that you need to apply indentations correctly - otherwise you get the error messages you experienced based on the yaml of your last post and the post before.

Note: never expect snippets to be at the correcte indention level your comple compose file needs.

I took your whole compose file and indented it correctly for you:

version: '3.9'

services:
  # Database
  db:
    image: mysql:8.0

    volumes:
      - db_data:/var/lib/mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: thepassword
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: thepassword
    networks:
      - wpnet
  # phpmyadmin
  phpmyadmin:
    depends_on:
      - db
    image: phpmyadmin/phpmyadmin
    restart: always
    ports:
      - '8023:80'
    environment:
      PMA_HOST: db
      MYSQL_ROOT_PASSWORD: thepassword
    networks:
      - wpnet
  # Wordpress
  wordpress:
    depends_on:
      - db
    image: wordpress:latest
    ports:
      - '8024:80'
    restart: always
    volumes:
      - html:/var/www/html
    environment:
      WORDPRESS_DB_HOST: db:3306
      WORDPRESS_DB_USER: wordpress
      WORDPRESS_DB_PASSWORD: thepassword
    networks:
      - wpnet
  backup:
    # In production, it is advised to lock your image tag to a proper
    # release version instead of using `latest`.
    # Check https://github.com/offen/docker-volume-backup/releases
    # for a list of available releases.
    image: offen/docker-volume-backup:latest
    restart: always
    environment:
      BACKUP_FILENAME: backup-%Y-%m-%dT%H-%M-%S.tar.gz
      BACKUP_LATEST_SYMLINK: backup-latest.tar.gz
    volumes:
      - db_data:/backup/my-app-backup:ro
      - html:/backup/my-app2-backup:ro
      # Mounting the Docker socket allows the script to stop and restart
      # the container during backup. You can omit this if you don't want
      # to stop the container. In case you need to proxy the socket, you can
      # also provide a location by setting `DOCKER_HOST` in the container
      - /var/run/docker.sock:/var/run/docker.sock:ro
      # If you mount a local directory or volume to `/archive` a local
      # copy of the backup will be stored there. You can override the
      # location inside of the container by setting `BACKUP_ARCHIVE`.
      # You can omit this if you do not want to keep local backups.
      - /root:/archive      

volumes:
  db_data:
    driver: local-persist
    driver_opts:
      mountpoint: /wordpress/db_data
  html:
    driver: local-persist
    driver_opts:
      mountpoint: /wordpress/html

farzadha2 · July 21, 2022, 3:55pm

Thank you so much, duly noted, i copied from what you posted

but im getting this error when i try to redeploy it
failed to deploy a stack: Service "db" uses an undefined network "wpnet" : exit status 1

meyay · July 21, 2022, 4:07pm

… please check the compose file of your first post and the compose file you provided as input that I used to fix the indention problem.

farzadha2 · July 23, 2022, 8:12pm

Thank you so much for the reply, sorry for the late reply, didn’t get an email alert.

Finally got it working with this

version: '3.9'

services:
  # Database
  db:
    image: mysql:8.0

    volumes:
      - db_data:/var/lib/mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: thepassword
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: thepassword
    networks:
      - wpnet
  # phpmyadmin
  phpmyadmin:
    depends_on:
      - db
    image: phpmyadmin/phpmyadmin
    restart: always
    ports:
      - '8023:80'
    environment:
      PMA_HOST: db
      MYSQL_ROOT_PASSWORD: thepassword
    networks:
      - wpnet
  # Wordpress
  wordpress:
    depends_on:
      - db
    image: wordpress:latest
    ports:
      - '8024:80'
    restart: always
    volumes:
      - html:/var/www/html
    environment:
      WORDPRESS_DB_HOST: db:3306
      WORDPRESS_DB_USER: wordpress
      WORDPRESS_DB_PASSWORD: thepassword
    networks:
      - wpnet
  backup:
    # In production, it is advised to lock your image tag to a proper
    # release version instead of using `latest`.
    # Check https://github.com/offen/docker-volume-backup/releases
    # for a list of available releases.
    image: offen/docker-volume-backup:latest
    restart: always
    environment:
      BACKUP_FILENAME: backup-%Y-%m-%dT%H-%M-%S.tar.gz
      BACKUP_LATEST_SYMLINK: backup-latest.tar.gz
    volumes:
      - db_data:/backup/db_data:ro
      - html:/backup/html:ro
      # Mounting the Docker socket allows the script to stop and restart
      # the container during backup. You can omit this if you don't want
      # to stop the container. In case you need to proxy the socket, you can
      # also provide a location by setting `DOCKER_HOST` in the container
      - /var/run/docker.sock:/var/run/docker.sock:ro
      # If you mount a local directory or volume to `/archive` a local
      # copy of the backup will be stored there. You can override the
      # location inside of the container by setting `BACKUP_ARCHIVE`.
      # You can omit this if you do not want to keep local backups.
      - ${HOME}/backups:/archive      
networks:
  wpnet:
volumes:
  db_data:
    driver: local-persist
    driver_opts:
      mountpoint: /tempwp/db_data
  html:
    driver: local-persist
    driver_opts:
      mountpoint: /tempwp/html

Topic		Replies	Views
Docker Backup and restore question General docker	7	2098	July 17, 2022
Docker Volume backup best practices General docker	6	36635	September 8, 2018
Combine 2 dockerfiles with docker compose (tomcat + postgresql) General	13	14437	July 29, 2019
Docker-compose.yml V3 and volumes General	0	3237	October 9, 2017
Typo in Docker userguide on volumes? General	0	1145	December 31, 2015

Question about backing up docker and restoring tests?

Related topics