Share and learn in the Docker community.
I had a Debian server running Docker and Portainer. The OS got compromised, but I did have a disk backup.
I took the opportunity to switch the system over to Proxmox, now I’m trying to restore all my Docker containers to the fresh install of a TurnKey Linux CT with Docker and Portainer installed.
I’ve extracted the /var/lib/docker folder, but I’m running into issues transferring it from Windows to the server. What data in that folder do I need to transfer, and what can I discard?
Thanks.
If you archived the whole docker data root then you need to copy everything to the new server and hope it will work when you extract the archive. It depends on how you archived it. You can’t extract just a part of it.
The backup was “dd full disk” I believe, from within OMV5. May have been the “dd” option instead.
Either way, the full directory listing of the OS partition is in the dd archive.
Admittedly, I don’t know much about this, but I’m surprised more than the overlay2, containers, etc folders are needed.
When I try to copy over I get these errors in Teracopy. Code 2 and Code 3.
Any ideas?
It looks like the files listed here are .symlink or .alias.
/var/lib/docker which is usually the Docker data root contains many files and symbolic links. If you don’t copy everything, meybe you will have the images, but not the references to the images so you will not be able to use any of them. Or maybe you will have the references and the images, but not the containers. You can also break Docker networks or the volumes. Everything in the dta root is there for a reason. You should never change it manually.
Yes, as I wrote above. That folder was on a Linux filesystem, you should use a Linux to extract it.
What’s the easiest way to do that? I’ve tried mounting the dd image, but I got an error.
Something like this should do the trick:
losetup --partscan --find --show {path/to/dd/image}
mount /dev/loop0p1 /mnt
Have you tried to google for “mount image created by dd”?
I tried something very similar to that, and a few different methods. I kept getting a “bad superblock” or “can’t read superblock” error message. Any ideas?
Clearly I am missing something. You had a Debian server, which is Linux. You want to move the data to an other Linux, but you can’t mount the image that you created with dd, but you could somehow open it on Windows. How did you do that? Do you remember the DD command that you used to backup the data?
Can you share those methods? When it comes to dd, I always need to do some research because I never remember how I should use it, but we may recognize a mistake in a command if we see it.
The dd archive was created by OMV and it’s built-in backup tool.
In Windows I’m able to open the dd archive with 7zip. I’m not sure but bad-superblock seems to be a data corruption issue, or when the archive isn’t created properly. I do know that the backup wasn’t a “full” disk image, so maybe that’s why I can’t mount it.
I don’t recall the commands I used, but I also tried your listed command and unfortunately wasn’t successful.
I did manage to get the archive extracted but all the symlinks are missing. When I browse the archive in Windows it shows the symlinks as shortcuts.
When I go to load the container, it tries to reference one of the symlinks in overlay2 and fails. Is there a way to recreate or refresh the overlay2 folder structure?
There is not. Though, you could throw away all damaged containers if you used volumes for each container. The only valuable data is stored in the volumes. The docker engine can be installed, the container can be recreated.
Normally, people would just backup the content of volumes (while the containers are stopped). During recovery they would install docker-ce from scratch, and
Oh interesting. I believe I did use volumes for all the containers. I was running OMV on bare-metal with the storage attached locally, and mapped the /config, /data, etc volumes to paths outside of the container. Is that what you mean?
Example below.
volumes:
- /srv/dev-disk-by-uuid-d18f5076-1a05-4068-85e3-5d09f5eb3757/Config/gotify:/app/data
All the config data is in one folder /srv/dev-disk......../Config/ so I think your 2nd option is the best route for me?
If I’ve got this right, I should nuke my current Docker install, start fresh. With my /Config folders and data in place, restore each container one by one manually from my backed up compose files?
Oh boy, I already wrote the answer, and overwrote it with content supposed for a different topic. Then I deleted the post, because I thought it was misplaced…
You are on the right path now! Just make sure you update the compose file to point to the new host paths where you restore the data into. Then restore all containers belonging to a compose file, until you restored all containers from all compose files.
With this approach, the worst thing that can happen is that you might need to fix ownership of the restored data (=align it with the UID/GID used by the containerized process).