How to make mariadb database changes persistent

General
1

Hi all.

I created 2 containers, one with a mariadb database server running, and another with Piwik Analytics installed. I start the mariadb container first and link to it from the Piwik Analytics container. So far so good. When I try to commit the mariadb container to a new image, the updates made to the mariadb container with the piwik database are NOT commited to the new image. So my question is how do I make a database persist when commiting a container with a new database to another new image, so I can run new containers using that database?

where the command to start the piwik-database is:

docker run -itd --name piwik-database --hostname piwik-database keithroberts/alpine-mariadb2

to run the container and link to the running piwik-database container:

$ docker run -itd --name ubuntu-piwik-v2 --hostname ubuntu-piwik-v2 --link piwik-database keithroberts/ubuntu-piwik-v2

vagrant@vagrant-ubuntu-trusty-64:~$ docker exec -it piwik-database sh
/ #
/ # mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 10.1.14-MariaDB MariaDB Server

Copyright © 2000, 2016, Oracle, MariaDB Corporation Ab and others.

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

MariaDB [(none)]> show databases;
±-------------------+
| Database |
±-------------------+
| information_schema |
| mysql |
| performance_schema |
| piwik_database |
| test |
±-------------------+
5 rows in set (0.00 sec)

When I commit this to a new image and then run another container from that new image, I loose the piwik_database:

vagrant@vagrant-ubuntu-trusty-64:~$ docker exec -it piwik-database-works sh
/ #
/ # mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 10.1.14-MariaDB MariaDB Server

Copyright © 2000, 2016, Oracle, MariaDB Corporation Ab and others.

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

MariaDB [(none)]> show databases;
±-------------------+
| Database |
±-------------------+
| information_schema |
| mysql |
| performance_schema |
| test |
±-------------------+
4 rows in set (0.00 sec)

MariaDB [(none)]>

So how do I keep the new piwik_database so I can reuse it again in other containers or images please?

TIA

Keith

2

one solution could be to use a volume to store your databases. That way it was actually independent from any of the containers.

where does MariaDB store it’s databases /var/lib/mysql? So once you put that folder into a volume (several approaches to do that exist) your databases created should persist in that volume.

That’s not really solving the riddle of the disappeared database but could be a workable solution (which might be even more flexible then what you had planned initially)

3

Thanks for your reply vrms. Mariadb stores it’s databases in /var/lib/mysql. When you create another container from an image that contains a default empty database, that’s what you get in the new container. Hence the ‘missing’ database.

Yes I think that’s the way to go with persistent container storage. Reading up on this I found:

Share Directories via Volumes
http://docker.readthedocs.org/en/v0.7.3/use/working_with_volumes/

New in version v0.3.0: Data volumes have been available since version 1 of the Docker Remote API

A data volume is a specially-designated directory within one or more containers that bypasses the Union File System to provide several useful features for persistent or shared data:

Data volumes can be shared and reused between containers. This is the feature that makes data volumes so powerful. You can use it for anything from hot database upgrades to custom backup or replication tools. See the example below.

Changes to a data volume are made directly, without the overhead of a copy-on-write mechanism. This is good for very large files.

Changes to a data volume will not be included at the next commit because they are not recorded as regular filesystem changes in the top layer of the Union File System

Each container can have zero or more data volumes.

So it looks like I might need to move the mariadb database to it’s own DATA container, and then tell the piwik-database container running the mariadb server application to use that as a shared volume?

This seems to be the best option at the moment, as it allows more than one container to share the same database cluster.

So I guess shared data volumes would be usefull for writing any persistent data such as log files, which could eventually use up the maximum number of image layers available to a container?

4

…and the standard images declare that as a VOLUME, so if you do nothing special, if you start a database container, do some activity on it, stop the container, and restart it again, you should see the same content (provided you don’t docker rm -v or docker volume rm to destroy the backing volume).

If you’re using Docker Compose, I think it’s a little more aggressive about cleaning up volumes, and you might be well-served to use the version 2 Compose file format (if you can use a new enough Docker stack for that) and separately declare the data volume.

The application shouldn’t need to access the files underneath the database, so you shouldn’t need to share the volume.

You can always have multiple containers talking to the same database server. Probably the best setup on a current Docker environment is:

  • Create a named data volume to hold the database contents
  • Create a private bridge network
  • Spin up a single MariaDB server, on the private network, using the named data volume
  • Spin up any number of applications, pointing them at mariadb:3306 for their database server, and relying on Docker’s internal DNS resolution to point them at the container named mariadb

That statement doesn’t quite make sense. I never use docker commit; while I do have quite a few private images, I build them with Dockerfiles and docker build. Shared data volumes are useful for things like log files, but mostly in that they need to be read by host administrators regularly who don’t want to have to dive into container space to do it.

For your particular situation, my application that needs a significant amount of preloaded database content works by having a script that starts up the database, loads the data, and then runs a mysql_dump. That dump file is packed into my application image, and when it starts up, an ENTRYPOINT script restores the dump if it’s required.