Docker Community Forums

Share and learn in the Docker community.

Only the first service port is getting exposed when doing a stack deploy in Docker swarm

I am doing a stack deploy using a docker-compose yml file as shown below:

[docker@docker-mdl-1 compose]$ docker version
Client:
 Version:      1.13.0
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:55:28 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.0
 API version:  1.25 (minimum version 1.12)
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:55:28 2017
 OS/Arch:      linux/amd64
 Experimental: false


[docker@docker-mdl-1 compose]$ more docker-compose-test.yml
version: '3'
services:
  testapp1:
    hostname: testapp1
    image: 1221-testapp
    environment:
      - ADMIN_PORT=7001
    ports:
      - "7001:7001"
    deploy:
      mode: replicated
      replicas: 1
      labels: [APP=TEST]
    networks:
      - testappnet

  testapp2:
    hostname: testapp2
    image: 1221-testapp
    environment:
      - ADMIN_PORT=8001
    ports:
      - "8001:8001"
    deploy:
      mode: replicated
      replicas: 1
      labels: [APP=TEST]
    networks:
      - testappnet

  testapp3:
    hostname: testapp3
    image: 1221-testapp
    environment:
      - ADMIN_PORT=9001
    ports:
      - "9001:9001"
    deploy:
      mode: replicated
      replicas: 1
      labels: [APP=TEST]
    networks:
      - testappnet

networks:
    testappnet:


$ docker stack deploy --compose-file docker-compose-test.yml TEST
Creating network TEST_testappnet
Creating service TEST_testapp1
Creating service TEST_testapp2
Creating service TEST_testapp3

However, only the port 7001 of the first service, testapp1 is getting exposed. Ports 8001 and 9001 of the other two services are not accessible.

[docker@docker-mdl-1 compose]$ docker node ls
ID                           HOSTNAME      STATUS  AVAILABILITY  MANAGER STATUS
0yi9334xjife9kfe0i86py5fp    docker-mdl-2  Ready   Active
bcpml15x6ryqaqwqfuye9nb35 *  docker-mdl-1  Ready   Active        Leader
c6uii92tbrsvuqvodwjwz0miq    docker-mdl-3  Ready   Active


[docker@docker-mdl-1 compose]$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
4a3wtg7u9dwy        TEST_testappnet     overlay             swarm
bc71aa6d1e64        bridge              bridge              local
42898b79ab60        docker_gwbridge     bridge              local
32fd3b078818        host                host                local
ef9jorbagzxw        ingress             overlay             swarm
8c18280b5b0f        none                null                local


[docker@docker-mdl-1 compose]$ docker service ls
ID            NAME           MODE        REPLICAS  IMAGE
6mxqoo9hbtyk  TEST_testapp2  replicated  1/1       1221-testapp
iae3jus2rtdo  TEST_testapp1  replicated  1/1       1221-testapp
r1qc7kglkjjh  TEST_testapp3  replicated  1/1       1221-testapp


[docker@docker-mdl-1 compose]$ docker service inspect TEST_testapp1 --pretty

ID:             iae3jus2rtdosjyp4on49xg38
Name:           TEST_testapp1
Labels:
 APP=TEST
 com.docker.stack.namespace=TEST
Service Mode:   Replicated
 Replicas:      1
Placement:
ContainerSpec:
 Image:         1221-testapp
 Env:           ADMIN_PORT=7001
Resources:
Networks: 4a3wtg7u9dwyrh8zwm5jk4pxb
Endpoint Mode:  vip
Ports:
 PublishedPort 7001
  Protocol = tcp
  TargetPort = 7001


[docker@docker-mdl-1 compose]$ docker network inspect TEST_testappnet
[
    {
        "Name": "TEST_testappnet",
        "Id": "4a3wtg7u9dwyrh8zwm5jk4pxb",
        "Created": "2017-01-26T08:16:28.935655168-08:00",
        "Scope": "swarm",
        "Driver": "overlay",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "10.0.0.0/24",
                    "Gateway": "10.0.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Containers": {
            "1ecfdfb989df8a7e35810c9bb3890b2824188e4712ab6446158077afed4db977": {
                "Name": "TEST_testapp2.1.r5sod1g5tnqmjuk8hmcyipuff",
                "EndpointID": "68a22636ff068080b107a5a7da7215dcaaaeecc54444b4d9d103c5c1326f80b2",
                "MacAddress": "02:42:0a:00:00:05",
                "IPv4Address": "10.0.0.5/24",
                "IPv6Address": ""
            }
        },
        "Options": {
            "com.docker.network.driver.overlay.vxlanid_list": "4096"
        },
        "Labels": {
            "com.docker.stack.namespace": "TEST"
        },
        "Peers": [
            {
                "Name": "docker-mdl-2-82658e710184",
                "IP": "xx.xx.xx.23"
            },
            {
                "Name": "docker-mdl-1-25cec51eb7cd",
                "IP": "xx.xx.xx.125"
            },
            {
                "Name": "docker-mdl-3-9aa775b39793",
                "IP": "xx.xx.xx.24"
            }
        ]
    }
]


[docker@docker-mdl-1 compose]$ docker service inspect TEST_testapp2 --pretty

ID:             6mxqoo9hbtyk756njpslu5pjd
Name:           TEST_testapp2
Labels:
 APP=TEST
 com.docker.stack.namespace=TEST
Service Mode:   Replicated
 Replicas:      1
Placement:
ContainerSpec:
 Image:         1221-testapp
 Env:           ADMIN_PORT=8001
Resources:
Networks: 4a3wtg7u9dwyrh8zwm5jk4pxb
Endpoint Mode:  vip
Ports:
 PublishedPort 8001
  Protocol = tcp
  TargetPort = 8001


[docker@docker-mdl-1 compose]$ docker service inspect TEST_testapp3 --pretty

ID:             r1qc7kglkjjhz3piz1sqpsmqv
Name:           TEST_testapp3
Labels:
 APP=TEST
 com.docker.stack.namespace=TEST
Service Mode:   Replicated
 Replicas:      1
Placement:
ContainerSpec:
 Image:         1221-testapp
 Env:           ADMIN_PORT=9001
Resources:
Networks: 4a3wtg7u9dwyrh8zwm5jk4pxb
Endpoint Mode:  vip
Ports:
 PublishedPort 9001
  Protocol = tcp
  TargetPort = 9001

I can access the application running on port 7001 from any of the 3 nodes. However, the port 8001 and 9001 are not responding on any of the nodes. I have verified these ports are responding within the respective containers and the published ports are also shown listening on all the nodes. However, the port binding from published port to target port is not working for ports 8001 and 9001. You can reproduce this problem using any application image with a port exposed and using a docker-compose.yml file as shown above.

[docker@docker-mdl-1 compose]$ netstat -an | grep 7001
tcp6       0      0 :::7001                 :::*                    LISTEN
                  LISTEN
[docker@docker-mdl-1 compose]$ netstat -an | grep 8001
tcp6       0      0 :::8001                 :::*                    LISTEN

[docker@docker-mdl-1 compose]$ netstat -an | grep 9001
tcp6       0      0 :::9001                 :::*                    LISTEN

I would expect all the 3 ports 7001, 8001 and 9001 should be working on all the nodes in swarm mode.
How can we fix this problem?

Update:

If I use the same ADMIN_PORT 7001(which is the default) on all the nodes and change the port mappings as shown below in docker-compose-test.yml it works fine.

version: '3'
services:
  testapp1:
    hostname: testapp1
    image: 1221-testapp
    environment:
      - ADMIN_PORT=7001
    ports:
      - "7001:7001"
    deploy:
      mode: replicated
      replicas: 1
      labels: [APP=TEST]
    networks:
      - testappnet

  testapp2:
    hostname: testapp2
    image: 1221-testapp
    environment:
      - ADMIN_PORT=7001
    ports:
      - "8001:7001"
    deploy:
      mode: replicated
      replicas: 1
      labels: [APP=TEST]
    networks:
      - testappnet

  testapp3:
    hostname: testapp3
    image: 1221-testapp
    environment:
      - ADMIN_PORT=7001
    ports:
      - "9001:7001"
    deploy:
      mode: replicated
      replicas: 1
      labels: [APP=TEST]
    networks:
      - testappnet

networks:
    testappnet:

With the above settings I can access the services using ports 7001, 8001 and 9001 from all the nodes in the swarm.

I have the following line in the Dockerfile which I used to build this image.

.....
.....
ENV ADMIN_PORT="${ADMIN_PORT:-7001}"
.....
.....
EXPOSE $ADMIN_PORT

The default value of ADMIN_PORT is set to 7001 in the Dockerfile. So it looks like the issue is, the environment variable, ADMIN_PORT which is set in the docker-compose-test.yml, is not being used when starting the container.