I am unable to get a docker swarm configured to accept HTTP connection to an nginx service.
I created 3 brand new Virtual Machines on my Proxmox install. None are running any firewalls via Proxmox or Ubuntu. Each VM has Ubuntu Server 24.04 installed with no further configuration and no firewall.
root@nuc-c1:~# ufw status
Status: inactive
The IP addresses are 10.0.0.231, 10.0.0.232, 10.0.0.233. I’ve completed an apt update, apt upgrade on each and rebooted. I’ve installed Docker as described by the official docker documentation. I can also run nginx on each node and access it with no problem. I’ve removed nginx after running this test.
I am able to set up the docker swarm (manager = 10.0.0.231) and the other two are workers (10.0.0.232, 10.0.232).
root@nuc-c1:~# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
ejtb52tjhotqo61rivm8tqty2 * nuc-c1 Ready Active Leader 28.3.2
uhq6mf9unpshfrk24w1ztp3hf nuc-c2 Ready Active 28.3.2
qhfj14hjlz8rceb3hkv8t7gp8 nuc-c3 Ready Active 28.3.2
I create an nginx service and scale it:
root@nuc-c1:~# docker service create --name nginx --publish published=80,target=80 nginx
h218git04bc1kagd9lm3xytgj
overall progress: 1 out of 1 tasks
1/1: running [==================================================>]
verify: Service h218git04bc1kagd9lm3xytgj converged
root@nuc-c1:~# docker service scale nginx=3
nginx scaled to 3
overall progress: 3 out of 3 tasks
1/3: running [==================================================>]
2/3: running [==================================================>]
3/3: running [==================================================>]
verify: Service nginx converged
root@nuc-c1:~# docker service ps nginx
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
vbxqrf6xwz4r nginx.1 nginx:latest nuc-c1 Running Running about a minute ago
c685thb5htjv nginx.2 nginx:latest nuc-c3 Running Running 38 seconds ago
tx61wknxm6s3 nginx.3 nginx:latest nuc-c2 Running Running 38 seconds ago
root@nuc-c1:~# docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2cb26a4ce621 nginx:latest "/docker-entrypoint.…" 14 minutes ago Up 14 minutes 80/tcp nginx.1.vbxqrf6xwz4rkdm38r0r9ahbs
When I try to access nginx from my laptop browser, my connection times out:
HTTP://10.0.0.231
I am seeing TCP traffic on 10.0.0.231 on port 80 when I use tcpdump:
** tcpdump -i ens18 dst port 80**
From 10.0.0.231:
I can curl from 10.0.0.231:
root@nuc-c1:~# curl http://10.0.0.231
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
But I can’t curl to any of the other nodes (times out):
curl HTTP://10.0.0.232
curl HTTP://10.0.0.233
Here is what the docker networks look like:
root@nuc-c1:~# docker network ls
NETWORK ID NAME DRIVER SCOPE
7297ed406809 bridge bridge local
67597c64fdd3 docker_gwbridge bridge local
5d484ccded12 host host local
dv00l5t3qcgq ingress overlay swarm
27801b5b1e48 none null local
And here is what the ingress network looks like:
root@nuc-c1:~# docker network inspect ingress
[
{
"Name": "ingress",
"Id": "dv00l5t3qcgq79mghlxcy8zz8",
"Created": "2025-07-18T12:50:59.894891478Z",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv4": true,
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "10.0.0.0/24",
"Gateway": "10.0.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": true,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"2cb26a4ce62142bc7124bd6d3d8e1eddd5ffec38dc6071ad4ac6d50e2a2c4732": {
"Name": "nginx.1.vbxqrf6xwz4rkdm38r0r9ahbs",
"EndpointID": "01667c27411f707ae88f6451bc6b41c7587dd72bf2ef9adab1df81ac9c38b995",
"MacAddress": "02:42:0a:00:00:ae",
"IPv4Address": "10.0.0.174/24",
"IPv6Address": ""
},
"ingress-sbox": {
"Name": "ingress-endpoint",
"EndpointID": "c71318a6646a31410b4384fec5a94f1b0066904f010a85eb97a39f5b22600f02",
"MacAddress": "02:42:0a:00:00:02",
"IPv4Address": "10.0.0.2/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "4096"
},
"Labels": {},
"Peers": [
{
"Name": "f7a8192a9430",
"IP": "10.0.0.231"
},
{
"Name": "167df137fe24",
"IP": "10.0.0.232"
},
{
"Name": "95c2e18cf098",
"IP": "10.0.0.233"
}
]
}
]
And the docker_gwbridge:
root@nuc-c1:~# docker network inspect docker_gwbridge
[
{
"Name": "docker_gwbridge",
"Id": "67597c64fdd389ffb7018d4b879bec3a5e08691c2d3cdef1d936120495ceacff",
"Created": "2025-07-17T21:07:09.559971447Z",
"Scope": "local",
"Driver": "bridge",
"EnableIPv4": true,
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.18.0.0/16",
"Gateway": "172.18.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"2cb26a4ce62142bc7124bd6d3d8e1eddd5ffec38dc6071ad4ac6d50e2a2c4732": {
"Name": "gateway_25632968d00c",
"EndpointID": "cfef0de29bc74db35f52ea81aaa82fa923a579174abcee65466e80894210113b",
"MacAddress": "8a:32:74:fe:31:18",
"IPv4Address": "172.18.0.3/16",
"IPv6Address": ""
},
"ingress-sbox": {
"Name": "gateway_ingress-sbox",
"EndpointID": "88dbfe8c29f304b4e52b247c9685149542c40abe3c59fc2ccf691ab902619266",
"MacAddress": "32:1c:f1:62:f9:65",
"IPv4Address": "172.18.0.2/16",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.bridge.enable_icc": "false",
"com.docker.network.bridge.enable_ip_masquerade": "true",
"com.docker.network.bridge.name": "docker_gwbridge"
},
"Labels": {}
}
]
FWIW, here is what nmap is showing:
root@nuc-c1:~# nmap 10.0.0.231
Starting Nmap 7.94SVN ( https://nmap.org ) at 2025-07-19 20:26 UTC
Nmap scan report for manager-1 (10.0.0.231)
Host is up (0.0000040s latency).
Not shown: 998 closed tcp ports (reset)
PORT STATE SERVICE
22/tcp open ssh
80/tcp filtered HTTP
root@nuc-c1:~# nmap 10.0.0.232
Starting Nmap 7.94SVN ( https://nmap.org ) at 2025-07-19 20:28 UTC
Nmap scan report for worker-1 (10.0.0.232)
Host is up (0.000030s latency).
Not shown: 998 closed tcp ports (reset)
PORT STATE SERVICE
22/tcp open ssh
80/tcp filtered http
MAC Address: BC:24:11:A8:E6:FC (Unknown)
Nmap done: 1 IP address (1 host up) scanned in 1.29 seconds
root@nuc-c1:~# nmap 10.0.0.233
Starting Nmap 7.94SVN ( https://nmap.org ) at 2025-07-19 20:28 UTC
Nmap scan report for worker-2 (10.0.0.233)
Host is up (0.00023s latency).
Not shown: 998 closed tcp ports (reset)
PORT STATE SERVICE
22/tcp open ssh
80/tcp filtered http
MAC Address: BC:24:11:56:0A:7B (Unknown)
Nmap done: 1 IP address (1 host up) scanned in 1.27 seconds
And finally, here is the iptables:
iptables -S
-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-BRIDGE
-N DOCKER-CT
-N DOCKER-FORWARD
-N DOCKER-INGRESS
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-FORWARD
-A DOCKER ! -i docker_gwbridge -o docker_gwbridge -j DROP
-A DOCKER ! -i docker0 -o docker0 -j DROP
-A DOCKER-BRIDGE -o docker_gwbridge -j DOCKER
-A DOCKER-BRIDGE -o docker0 -j DOCKER
-A DOCKER-CT -o docker_gwbridge -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-FORWARD -j DOCKER-INGRESS
-A DOCKER-FORWARD -j DOCKER-CT
-A DOCKER-FORWARD -j DOCKER-ISOLATION-STAGE-1
-A DOCKER-FORWARD -j DOCKER-BRIDGE
-A DOCKER-FORWARD -i docker_gwbridge -o docker_gwbridge -j DROP
-A DOCKER-FORWARD -i docker_gwbridge ! -o docker_gwbridge -j ACCEPT
-A DOCKER-FORWARD -i docker0 -j ACCEPT
-A DOCKER-INGRESS -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER-INGRESS -p tcp -m tcp --sport 80 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-INGRESS -j RETURN
-A DOCKER-ISOLATION-STAGE-1 -i docker_gwbridge ! -o docker_gwbridge -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o docker_gwbridge -j DROP
I have no idea why this may not be working, I am sure that I did something basic incorrectly because there is no way that this is so broken. Please let me know if you see something or need more information on anything.