Best practize full backup and restore ( image + volumes + config )

General Discussions
1

Hi,

i spend now some time with this topic, but did not really find a good solution.

The goal would be to simply backup all docker related information on host A and move it to host B.

Unfortunately it seems that docker ( why ever ) does not provide any solution for this.

You can export/commit/save/load/import/what ever volumes and images. Great.

But then you have an image and you have volumes. That will not bring back your docker config ( port mappings and so on ). For this you would need a docker compose ( which is not part of the backup ).

So you have the raw data and either you have the old docker compose file or you will have fun.

And what actually happens if you dont have named volumes? You have fun.

Is there really no script/software that will simply run a docker inspect, travers through all relevant data and just generate / save the required information ( image + volumes + config )?

Its nice if you have to consume your time to puzzle all together, but well, maybe its not necessary.

So is there anything out there, that will simply make a backup that you can restore it and just start your container(s) like you started them on the system where they came from?

Thank you!

2

The concept of container is to start them, run them, then “throw away”.

You usually would not go into a container, change things and back them up. Therefore no such tooling is even needed.

We have a docker-compose.yml that contains everything. If the container really needs to write files, we use a bind mount.

So everything is regular files, we can backup and move to different node, start new container there.

If you really need to backup your whole system, you could run them in a VM, tools like ProxMox enables easy migration of the whole VM to a new node.

3

While that may be true for the container, the docker volume (as the alternative to a bind mount) will have perishable data that needs to be backed up. This along with the current live config file for the docker container should be backed up.

Like the OP, I myself have been looking for a ‘one-stop-shop’ to backup current yaml config and associated volumes. Automated restore would be nice but not absolutely necessary.

There exists tools that do each of these things independently but I have yet to find any that does both and can be automated.

4

A local volume is barely different from a bind mount, except a volume is populated with container data when starting the container for the first time. You can customize the default host path of the volume and mix the advantages of both the volumes and bind mounts. When you remove such a volume, the actual data is not removed. The volume folder managed by Docker is just a link to the original data.

5

I appreciate the response, but this does little to answer the original questions. A local volume may be ‘barely’ different from a bind mount, but different it is. And regardless, I am not asking ‘what’ a volume is. I am asking ‘for’ a program, if it exists, that can do what I am asking.

I personally am not interested in backing up the image, as it is created when the docker-compose.yml is parsed. All I am interested in is a program that can stop or pause the container, build a live docker-compose file, and copy any associated volumes to a location of my choosing.

6

Because I replied to your statement about volumes. The original question was answered by @bluepuma77
Every question could be asked by adding “with Docker containers”, but ultimately this topic was about backup tools assuming Docker would require a special tool to backup your data. It all depends on the application. So you need a tool that can backup your filesystem when it is enough and follow the recommended, officially documented ways to backup relational databases. For backing up the filesystem, someone just uses rsync in a cronjob, but you could find tools like UrBackup or Duplicati. I’m sure you are familiar with some backup tools, so if you mount data from a fodler outide of the Docker data root where local volumes are stored by default and where it is not recommended to do anything without Docker, you can still use normal backup tools. Regarding stopping containers before starting backup, I don’t know about any special tool. Before starting the backup in a cronjob, you can run a command to stop the container if necessary. But lets say you have a Kubernetes or Docker Swarm cluster where services are restarted when stopped. So you could not justs top them. Some filesystems / volume managers support snapshots like LVM, ZFS or BTRFS so you can make a snapshot and copy it somewhere.

Special backup tools exist for Kubernetes for example like Velero or Portworx and more, but there you would have more things to backup for each application including config maps stored in the central etcd datatabase.

Yes, but with the custom host path, the difference is not really relevant. If you disagree, please, share what you think. You could be right, but I can’t think of any relevant difference now.