I am doing a stack deploy using a docker-compose yml file as shown below:
[docker@docker-mdl-1 compose]$ docker version
Client:
Version: 1.13.0
API version: 1.25
Go version: go1.7.3
Git commit: 49bf474
Built: Tue Jan 17 09:55:28 2017
OS/Arch: linux/amd64
Server:
Version: 1.13.0
API version: 1.25 (minimum version 1.12)
Go version: go1.7.3
Git commit: 49bf474
Built: Tue Jan 17 09:55:28 2017
OS/Arch: linux/amd64
Experimental: false
[docker@docker-mdl-1 compose]$ more docker-compose-test.yml
version: '3'
services:
testapp1:
hostname: testapp1
image: 1221-testapp
environment:
- ADMIN_PORT=7001
ports:
- "7001:7001"
deploy:
mode: replicated
replicas: 1
labels: [APP=TEST]
networks:
- testappnet
testapp2:
hostname: testapp2
image: 1221-testapp
environment:
- ADMIN_PORT=8001
ports:
- "8001:8001"
deploy:
mode: replicated
replicas: 1
labels: [APP=TEST]
networks:
- testappnet
testapp3:
hostname: testapp3
image: 1221-testapp
environment:
- ADMIN_PORT=9001
ports:
- "9001:9001"
deploy:
mode: replicated
replicas: 1
labels: [APP=TEST]
networks:
- testappnet
networks:
testappnet:
$ docker stack deploy --compose-file docker-compose-test.yml TEST
Creating network TEST_testappnet
Creating service TEST_testapp1
Creating service TEST_testapp2
Creating service TEST_testapp3
However, only the port 7001 of the first service, testapp1 is getting exposed. Ports 8001 and 9001 of the other two services are not accessible.
[docker@docker-mdl-1 compose]$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
0yi9334xjife9kfe0i86py5fp docker-mdl-2 Ready Active
bcpml15x6ryqaqwqfuye9nb35 * docker-mdl-1 Ready Active Leader
c6uii92tbrsvuqvodwjwz0miq docker-mdl-3 Ready Active
[docker@docker-mdl-1 compose]$ docker network ls
NETWORK ID NAME DRIVER SCOPE
4a3wtg7u9dwy TEST_testappnet overlay swarm
bc71aa6d1e64 bridge bridge local
42898b79ab60 docker_gwbridge bridge local
32fd3b078818 host host local
ef9jorbagzxw ingress overlay swarm
8c18280b5b0f none null local
[docker@docker-mdl-1 compose]$ docker service ls
ID NAME MODE REPLICAS IMAGE
6mxqoo9hbtyk TEST_testapp2 replicated 1/1 1221-testapp
iae3jus2rtdo TEST_testapp1 replicated 1/1 1221-testapp
r1qc7kglkjjh TEST_testapp3 replicated 1/1 1221-testapp
[docker@docker-mdl-1 compose]$ docker service inspect TEST_testapp1 --pretty
ID: iae3jus2rtdosjyp4on49xg38
Name: TEST_testapp1
Labels:
APP=TEST
com.docker.stack.namespace=TEST
Service Mode: Replicated
Replicas: 1
Placement:
ContainerSpec:
Image: 1221-testapp
Env: ADMIN_PORT=7001
Resources:
Networks: 4a3wtg7u9dwyrh8zwm5jk4pxb
Endpoint Mode: vip
Ports:
PublishedPort 7001
Protocol = tcp
TargetPort = 7001
[docker@docker-mdl-1 compose]$ docker network inspect TEST_testappnet
[
{
"Name": "TEST_testappnet",
"Id": "4a3wtg7u9dwyrh8zwm5jk4pxb",
"Created": "2017-01-26T08:16:28.935655168-08:00",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "10.0.0.0/24",
"Gateway": "10.0.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Containers": {
"1ecfdfb989df8a7e35810c9bb3890b2824188e4712ab6446158077afed4db977": {
"Name": "TEST_testapp2.1.r5sod1g5tnqmjuk8hmcyipuff",
"EndpointID": "68a22636ff068080b107a5a7da7215dcaaaeecc54444b4d9d103c5c1326f80b2",
"MacAddress": "02:42:0a:00:00:05",
"IPv4Address": "10.0.0.5/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "4096"
},
"Labels": {
"com.docker.stack.namespace": "TEST"
},
"Peers": [
{
"Name": "docker-mdl-2-82658e710184",
"IP": "xx.xx.xx.23"
},
{
"Name": "docker-mdl-1-25cec51eb7cd",
"IP": "xx.xx.xx.125"
},
{
"Name": "docker-mdl-3-9aa775b39793",
"IP": "xx.xx.xx.24"
}
]
}
]
[docker@docker-mdl-1 compose]$ docker service inspect TEST_testapp2 --pretty
ID: 6mxqoo9hbtyk756njpslu5pjd
Name: TEST_testapp2
Labels:
APP=TEST
com.docker.stack.namespace=TEST
Service Mode: Replicated
Replicas: 1
Placement:
ContainerSpec:
Image: 1221-testapp
Env: ADMIN_PORT=8001
Resources:
Networks: 4a3wtg7u9dwyrh8zwm5jk4pxb
Endpoint Mode: vip
Ports:
PublishedPort 8001
Protocol = tcp
TargetPort = 8001
[docker@docker-mdl-1 compose]$ docker service inspect TEST_testapp3 --pretty
ID: r1qc7kglkjjhz3piz1sqpsmqv
Name: TEST_testapp3
Labels:
APP=TEST
com.docker.stack.namespace=TEST
Service Mode: Replicated
Replicas: 1
Placement:
ContainerSpec:
Image: 1221-testapp
Env: ADMIN_PORT=9001
Resources:
Networks: 4a3wtg7u9dwyrh8zwm5jk4pxb
Endpoint Mode: vip
Ports:
PublishedPort 9001
Protocol = tcp
TargetPort = 9001
I can access the application running on port 7001 from any of the 3 nodes. However, the port 8001 and 9001 are not responding on any of the nodes. I have verified these ports are responding within the respective containers and the published ports are also shown listening on all the nodes. However, the port binding from published port to target port is not working for ports 8001 and 9001. You can reproduce this problem using any application image with a port exposed and using a docker-compose.yml file as shown above.
[docker@docker-mdl-1 compose]$ netstat -an | grep 7001
tcp6 0 0 :::7001 :::* LISTEN
LISTEN
[docker@docker-mdl-1 compose]$ netstat -an | grep 8001
tcp6 0 0 :::8001 :::* LISTEN
[docker@docker-mdl-1 compose]$ netstat -an | grep 9001
tcp6 0 0 :::9001 :::* LISTEN
I would expect all the 3 ports 7001, 8001 and 9001 should be working on all the nodes in swarm mode.
How can we fix this problem?
Update:
If I use the same ADMIN_PORT 7001(which is the default) on all the nodes and change the port mappings as shown below in docker-compose-test.yml it works fine.
version: '3'
services:
testapp1:
hostname: testapp1
image: 1221-testapp
environment:
- ADMIN_PORT=7001
ports:
- "7001:7001"
deploy:
mode: replicated
replicas: 1
labels: [APP=TEST]
networks:
- testappnet
testapp2:
hostname: testapp2
image: 1221-testapp
environment:
- ADMIN_PORT=7001
ports:
- "8001:7001"
deploy:
mode: replicated
replicas: 1
labels: [APP=TEST]
networks:
- testappnet
testapp3:
hostname: testapp3
image: 1221-testapp
environment:
- ADMIN_PORT=7001
ports:
- "9001:7001"
deploy:
mode: replicated
replicas: 1
labels: [APP=TEST]
networks:
- testappnet
networks:
testappnet:
With the above settings I can access the services using ports 7001, 8001 and 9001 from all the nodes in the swarm.
I have the following line in the Dockerfile which I used to build this image.
.....
.....
ENV ADMIN_PORT="${ADMIN_PORT:-7001}"
.....
.....
EXPOSE $ADMIN_PORT
The default value of ADMIN_PORT is set to 7001 in the Dockerfile. So it looks like the issue is, the environment variable, ADMIN_PORT which is set in the docker-compose-test.yml, is not being used when starting the container.