Swarm services not visible to traefik reverse proxy

Open Source Projects Swarm
1

I am currently evaluating docker for containerizing our DevOps infrastructure (jenkins, nexus, …) and I am building a prototyp using docker swarm. My current goal is to deploy two services for nexus repository manager and jenkins build server and a third service for a traefik reverse proxy / load balancer to make my other services available for users.

I created a traefik-compose.yml file for the traefik service like this:

version: "3.7"

services:
   proxy:
      image: traefik
      command: --api --docker --docker.swarmMode --docker.watch
      ports:
         - 80:80 
         - 8080:8080
      networks:
         - devops-net
      volumes:
         - /var/run/docker.sock:/var/run/docker.sock
      configs:
         - source: traefik.config
           target: /etc/traefik/traefik.toml
      deploy:
         placement:
            constraints:
               - node.role == manager
      labels:
         - "traefik.docker.network=devops-net"

networks:
   devops-net:
      external: true

configs:
   traefik.config:
      external: true

To start the service, I use

docker stack deploy --compose-file traefik-compose.yml traefik

Next I use my custom image bm/nexus to create a nexus service. And here my problem starts.
When I start a nexus service with docker service create my nexus service becomes visible to the traefik dashboard on localhost:8080 and I can access my nexus UI via localhost/nexus.

docker service create --name nexus --publish 8081:8081 --label traefik.port=8081 --label traefik.frontend=nexus --label traefik.frontend.rule=PathPrefix:/nexus --label traefik.backend=nexus --label traefik.docker.network=devops.net --label traefik.enable=true --network devops-net bm/nexus

I only publish port 8081 to check if my nexus service works.
When I start the nexus like this, it works and is visible to traefik dashboard, no matter on which swarm node it is started (I checked that about 100 times).

For better usability I wanted to put that long docker service create line into a separate nexus-compose.yml file for a nexus service like this:

version: "3.7"

services:
   nexus:
      image: bm/nexus
      networks:
         - devops-net
      deploy:
         placement:
            constraints:
               - node.role == worker
      labels:
         - "traefik.docker.network=devops-net"
         - "traefik.enable=true"
         - "traefik.port=8081"
         - "traefik.frontend=nexus"
         - "traefik.frontend.rule=PathPrefix:/nexus"
         - "traefik.backend=nexus" 
         - "traefik.docker.network=devops.net"

networks:
   devops-net:
      external: true

And I start this service using

docker stack deploy --compose-file nexus-compose.yml nexus

this creates a service nexus_nexus, but it is not visible to the traefik dashboard on localhost:8080 and I can NOT access my nexus UI via localhost/nexus. But if I add the published port 8081:8081 to the nexus-compose.yml file, I can access my nexus UI via localhost:8081, so the service sems to work.

I would be very happy if someone could help me with this, because I am wasting hours and hours and I don’t find a solution to this. Maybe I understand something wrong.

Here is some additional info that may help:
OS: Debian stretch
Docker version: 18.09.6

Thank you all in advance

2

Update:
Even if I add the nexus service form my nexus-compose.yml file to the traefik-compose.yml file and start both services in the same stack, my nexus service will not be visible to the traefik dashboard. I tried this because I thought my problem is maybe caused by the services not beig in the same stack.

My network (external) is an overlay network created with

docker network create --driver overlay devops-net

My traefik.config (external) is in the following config.toml file:

logLevel="DEBUG"
debug=true

[web]
address=":8080"

[docker]
endpoint="unix://var/run/docker.sock"
watch=true
swarmMode=true

Update 2:
I have it working now but I am not really sure what was the problem before. But as I have a working configuration now, I wanted to share this with the community, because it may help someone else in the future.
I divided my three services (traefik, nexus, jenkins) in three different compose files:

traefik-compose.yml:

version: "3.7"

services:
   proxy:
      image: traefik
      command: 
         - "--api" 
         - "--docker"
         - "--docker.swarmMode"
         - "--docker.watch"
         - "--docker.domain=mydomain.com"
         - "--docker.endpoint=unix://var/run/docker.sock"
         - "--entrypoints=Name:http Address::80 Redirect.EntryPoint.https"
         - "--entrypoints=Name:https Address::443 TLS"
         - "--defaultentrypoints=http,https"
      ports:
         - 80:80 
         - 8080:8080
      networks:
         - devops-net
      volumes:
         - /var/run/docker.sock:/var/run/docker.sock
      deploy:
         placement:
            constraints:
               - node.role == manager
      labels:
         - "traefik.docker.network=devops-net"

networks:
   devops-net:
      driver: overlay
      external: true

nexus-compose.yml:

version: "3.7"

services:
   nexus:
      image: bm/nexus
      ports:
         - 8081:8081
      networks:
         - devops-net
      deploy:
         placement:
            constraints:
               - node.role == worker
      labels:
         - "traefik.docker.network=devops-net"
         - "traefik.enable=true"
         - "traefik.port=8081"
         - "traefik.frontend=nexus"
         - "traefik.frontend.rule=PathPrefix:/nexus"
         - "traefik.backend=nexus"

networks:
   devops-net:
      driver: overlay
      external: true

jenkins-compose.yml:

version: "3.7"

services:
   jenkinsmaster:
      image: bm/jenkinsmaster
      environment:
         - ENV=dev
         - JENKINS_OPTS=--httpPort=8082 --prefix=/jenkins
      ports:
         - 8082:8082
      networks:
         - devops-net
      volumes:
         - /var/run/docker.sock:/var/run/docker.sock
         - type: bind
           source: ./jenkins_home
           target: /var/jenkins_home
      deploy:
         placement:
            constraints:
               - node.role == worker
      labels:
         - "traefik.docker.network=devops-net"
         - "traefik.enable=true"
         - "traefik.port=8082"
         - "traefik.frontend=jenkins"
         - "traefik.frontend.rule=PathPrefix:/jenkins"
         - "traefik.backend=jenkins"

networks:
   devops-net:
      driver: overlay
      external: true

If someone finds the explanation, what was my mistake before and what I did right now, please let me know. I will check this topic again and see if someone finds an answer.

3

Traefik in swarm mode required the labels to be a child of “deploy” and not the service.

Also i am quite surprised that your configuration works, since you forward http traffic to https, but only publish the ports for http and the dashboard.

services:
  nexus:
    image: bm/nexus
    ports:
      - 8081:8081
    networks:
      - devops-net
    deploy:
      placement:
        constraints: 
          - node.role == worker
      labels:
        - "traefik.docker.network=devops-net"
        - "traefik.enable=true"
        - "traefik.port=8081"
        - "traefik.frontend=nexus"
        - "traefik.frontend.rule=PathPrefix:/nexus"
        - "traefik.backend=nexus"

networks:
  devops-net:
    driver: overlay
    external: true

The parameters you expose as a command, I prefer to expose as a toml configuration. Though, i run mine as a global service and use consul to store letsencrypt certificates.

4

In your labels, you listed traefik.docker.network twice. once as ‘devops-net’ and again as ‘devops.net’. I think that may have been your problem.