Docker Community Forums

Share and learn in the Docker community.

How to backup and sync data from docker container

build
docker

(Vmdigitalnl) #1

The Context

I am developing a Symfony application where users can upload there own pictures, documents and more. I want to deploy this application with docker using 3 containers (nginx, php-fpm and mysql) on a VPS (Digital Ocean droplet). The way I would like to do this is creating the image locally, pushing it to the register, pulling form the server and run the install script.

 

Problems with redeploying and making backups:

1.How to back up the full project directory that resides in the php-fpm container (the source code and the user uploaded files) to another directory on the VPS?

2.How to share newly uploaded documents from the php-fpm container with the Nginx container?

3.How to can I make sure the user uploaded files are available to the application after redeploying?

 

Solutions I am considering
I am trying to come up with a good solution for the above problems for the past week and this is what I have thought of so far:

 

Current solution to problem 1:

Pulling all the code via git on the host and then using a bind-mount to make the data available in the docker container. However, this goes against the whole principle of being able to spin up the docker image anywhere independent of the host.

 

Current solution to problem 2 and 3: Using volumes

This solution however requires me to setup new volumes if someone on the team creates a new upload directory. If the team member does not communicate the new directory (or doesn’t know that he has to communicate it) people will upload files and the files will then not be there anymore when a new version is deployed.

 

I really have the feeling that there are better solutions possible and would like to ask you guys if you know solutions to the problems above.


(David Maze) #2

You can write a simple Dockerfile that installs your application in a new image. It’s common to check this in alongside the rest of your source code. Then when you change your application, test it locally; git commit && git push; docker build && docker push; then change the version tag on your deployed application and redeploy the containers.

It sounds like you’re on the right path here.

The “someone on the team creates a new upload directory” path sounds a little odd to me; I’d typically expect there’d be only one root directory for your application content. If you’re routinely adding directories that get mounted into your container, listing them all in a single file (and version controlling it) will help you get some reproducibility. If you haven’t looked at Docker Compose, its YAML file format will let you specify both the current image you want to use and any volumes you need mounted into the container, and running docker-compose up will apply any differences from the current state.

This is also a place where, as long as there have been named volumes, Docker has encouraged them, but contrary to the documentation, I think they’re harder to use. Using a host directory for content like this means you can use your existing backup tools and just globally ignore /var/lib/docker (good practice in any case).

Docker doesn’t really have any silver bullets to offer in terms of making it easy to move the content across hosts, or managing frequently-changing sets of volume mounts.

In short:

… Add a Dockerfile to your source repository, and build an image out of your application; don’t bind-mount your application code into the running container
… Add a docker-compose.yml to your source repository and use it to track the currently-running image and host directories that need to be mounted
… Use host directories to store the uploaded content, especially if you already know how to audit them and back them up


(Vmdigitalnl) #3

Thank you so much for the extensive reply! I have implemented your advise and have that working for now. Also someone else suggested me to look in to using separate objectstore (Digital Ocean spaces OR AWS S3) so that will be my next step.

Anyway thanks for your help!