How to handle server reboot when using docker-compose?

Docker Engine Compose
1

When using docker run for a single container, I understand that I can assign a restart policy to make sure the container is restarted by docker when the server reboots.

What is the equivalent practice when using docker-compose for all services defined by the docker-compose.yml?

If I stand up several services as containers using docker-compose, what is controlling the restart/reboot condition for everything as a whole and individually?

1 Like
2

Have been wondering about this myself. Turns out after a few false starts Google is our friend, and that it’s simply part of the compose file syntax:

https://docs.docker.com/v1.7/compose/yml/#working-dir-entrypoint-user-hostname-domainname-mem-limit-privileged-restart-stdin-open-tty-cpu-shares-cpuset-read-only

Further on setting up a production environment:

3
  1. That does answer on a per service/container, sort of 
 that part of the documentation doesn’t give a clear example of where to use those parameters. It just lists the parameters as if they are floating around in the file “somewhere”.

and more importantly

  1. That doesn’t explain how to have the whole definition in the compose restart itself in the same way as when docker-compose up was run originally. There are things that defined only in the docker-compose.yml file. How are those handled?

For one example:

What is controlling the depends_on references to make sure the services/containers that are depended on start first and then the ones that depend on start after but not before - as is defined in the docker-compose.yml but is not defined for the container on its own.

4

On 1) I would expect the restart: to be nested under the service/container you want to set it for. So the example given in the depends_on section of the compose reference could be:

version: '2'
services:
  web:
    build: .
    restart: always         # <--- my addition 
    depends_on:
      - db
      - redis
  redis:
    image: redis
  db:
    image: postgres

Does that not work for you if you test out a similar addition in your environment?

Regarding 2) I would simply expect declarations as depends_on: to work just as it did before you add restart: So in the example given above a natural consequence after a restart would be that the dependent services db and redis will be started before web. Something similar should implicitly be true for volumes and networks your services refer to.

6

Sorry to necro this thread, but I needed an answer to this very question.

After trying this, I found I had to add restart: always to the redis: section as well to make sure it came up. With just depends_on: redis and restart: always in the web: section, the apache container would restart after a reboot, but redis wouldn’t.

1 Like
7

No worries. I was wondering where to put that as wall. If I understand you correctly, you have to put “restart: always” in each service that you want to start on system boot?

I wasn’t sure if you could make a single “global” entry in the file that would applied to all of the services or not.

For example, the top of my file looks like this before I add restart: always to it

version: “3.6”
services:
web:

web parameters
database:

database parameters

Then with the restart: always parameter

version: “3.6”
restart: always ← here at top level?
services:
restart: always ← or here at services level ?
web:

web parameters
restart: always ← or perhaps in each service, here?

database:

database parameters
restart: always ← or perhaps in each service, here?

8

With docker-compose, the “restart:” node is a direct child node of a specific service, a sibling of “image:”
For Swarm stack deploymets, the “restart_policy:” node is a child node of “deploy:”, which itself is a sibling of “image:”

9

The problem with the restart policy at the service level is that docker itself is unaware of the depends_on between services that only appears in the docker-compose.yml file. docker-compose up will start services in dependency order and can wait on services to be healthy before starting their dependent services. But on reboot, every service that is marked for restart will be immediately started without waiting for their dependency services to be healthy first.
Is there a way to address that? It seems incomplete to have depends_on functionality if it only works in an explicit up call but not on reboot.

3 Likes
10

This is exactly what I am looking for. Is there any solution for this yet?

11

Hey, more than a year later, is there an answer to this? :sweat_smile:
I’m havin the exact same problem, I’m trying to start two containers after one another importing variables via the docker compose file and I need to restart them after a reboot/power outage


Greetings Luis

12

Docker Compose is a kind of client for Docker. Features that Docker doesn’t support are used for developement and in some cases to make sure the first started container can initialize data which will be mounted by another container as well. So when you first start the container, the order of the containers are more important. I never used the depends_on feature with healthchecks as the applications should handle when a dependency is not available. If the app fails because it can’t access an endpoint and the container restarts, it can try again. Or endpoint checking can be added in a loop in the entrypoint before the application starts and run until the endpoint is ready. This works regardless of how where your containers are running (Kubernetes, Swarm, Compose).

13

There is more subtle problem. Consider you have two containers A and B:

A is a regular container.
B is a container with --network=container:A (in compose it would be network_mode: service:A).

Docker starts A and B in random order. If B happens to start first, it will fail with “cannot join network of a non running container:A” and compeltely ignore --restart=always because docker considers this as a configuration error.
This is quite a pain as currently there is no way to resolve this, except having some script that will check for such condition.

1 Like
14

That’s a good point. Thanks for commenting. I don’t remember if I ever needed sharing the network between containers or anything else in production where auto start was needed at reboot. I used it in CI/CD pipelines and to connect to an existing container when running a scheduled job container or running one manually so I never actually saw that error message, but you are right.

In this case I would probably try a Systemd service that just executes the docker compose up command. You can probably find existing guides to do tha.

As far as I know, even Docker Swarm doesn’t support sharing network namespaces, but you can ask for supporting this case with the Docker Engine in the roadmap, so then it can be supported in Docker Compose as well.

If it is not possible in Docker Engine (using docker run commands), Docker Compose cannot help as a client application.

15

It does. It requires service:<servicename> instead of container:<containername>.

16

Docker compose supports the network namespace sharing, but Docker Swarm says network_mode is ignored as it is not supported. The quoted part of my message indeed looks like I was writing about the namespace sharing, but I just wanted to note that although Docker Swarm would be something I would try when something is not working in Compose, but it doesn’t seem to support the parameter, so it couldn’t be used instead of compose to handle shared network namespaces when starting containers in a stack at boot.

Did I miss something, or I just confused you with my previous message? :slight_smile:

17

I must have remembered it wrong :slight_smile: It a makes sense that it’s not supported. How should a container on another node use the same network namespace like a container on a different node