ERROR: error while removing network: network docker_default

Docker Engine Compose
1

I am setting up a CI pipeline and using Compose to start and stop services. There are a few services that are not a part of the pipeline that have been started by a separate compose file. When I stop the services that are in the pipeline, I get the error message in the subject line:

ERROR: error while removing network: network docker_default id 1396097fbd6632558c746a5b539f2e79d59e5d1ce4b8b0b3870ef0bd174b9252 has active endpoints

In my case, Compose creates a docker_default network when services in the docker-compose-1.yml file start. These are services like my API gateway, service registry, etc. When the services in the docker-compose-2.yml file are started, Compose automatically joins them to the docker_default network just fine. However, shutting down the services in the docker-compose-2.yml file yields the error above and fails the build.

Is there a way to tell Compose not to attempt to shut down the network when doing docker-compose down? I do not see an argument that would allow for that.

Client: Docker Engine - Community
Version: 18.09.2
API version: 1.39
Go version: go1.10.8
Git commit: 6247962
Built: Sun Feb 10 04:12:39 2019
OS/Arch: darwin/amd64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 18.09.2
API version: 1.39 (minimum version 1.12)
Go version: go1.10.6
Git commit: 6247962
Built: Sun Feb 10 04:13:06 2019
OS/Arch: linux/amd64
Experimental: false

2

This is a private docker network belonging to the compose stack and to docker-compose state.
If you need a network with a decoupled lifecylce, you have to create an external network with the cli, declare it in your compose,yml as external and use it in your services.

See: https://docs.docker.com/engine/reference/commandline/network_create/

3

Thank you for the info, Metin! Is that how it normally works in a CI/CD pipeline? Does that cause any issues with simultaneous/parallel builds?

4

Also, this approach seems to report the services from the docker-compose-1.yml file as orphaned services:

WARNING: Found orphan containers (docker_api-gateway_1, docker_registry_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.

5

Of course an externaly created network has advantages and drawbacks:

advantage:
– it is decouple from docker-compose or swarm stack lifecycles
– you can pick your own prefix-free name

disadvantage:
– each instance of the compose/swarm stack will require its own external network
– the portability of the compose/swarm stack is compromised, as you need to create the external network before the stack can be startet

I never used such a setup on a docker-compose stack. With swarm, you can attach as many services (standalone or from a swarm stack) to an attachable external overlay network.

What I remember from the good old docker-compose days:
If the network is created as a private network from within the compose.yml, its name will be {foldername}_{networkname}, if you have a second compose.yml in the same folder that declares the same network name, you will have a conflict, as each docker-compose.yml “thinks” that its state reflects the world. Are you sure that you created the network on the cli and added external:true underneath the declared network?

6

Hi Metin,

I think a way to work around the network issue would be to dynamically create the name of the network in each CI build. That’s pretty easy to do using environment variables on the command line. However, it seems that Compose files does not correctly expand variables defined in a network name.

In the Compose file, I have defined the network as:

networks: 
  ${NET}:
    external: true

Running docker-compose -f docker-compose-ci.yml config does not show any errors. Here’s the abbreviated output of that command:

  postgresql:
    environment:
      POSTGRES_PASSWORD: ''
      POSTGRES_USER: test
    image: postgres:11.3
    networks:
      app_network_1: null
version: '3.7'

So, it shows the network being read from the environment variable. And I can see that network when I docker network ls, but Compose complains and fails to start the services:

$ docker-compose -f docker-compose-ci.yml up -d
WARNING: Some networks were defined but are not used by any service: ${NET}
ERROR: Service "app" uses an undefined network "app_network_1"

So, theoretically, it should work – but something is missing or not happening correctly here(?) or PEBKAC.

On a side note, you mentioned the “good old docker-compose days.” What are you using now instead of Compose to manage the dependencies?

Thanks!

7

I can realy help you with the variable substitution. As maintaining docker-compose.yml files is a nightmare, we started to use templating to render deployment specific docker-compose.yml files.

Docker Swarm for time beeing, though for me it is inevitable to migrate to Kubernetes in the near future.