I’m using a mixed-platform (linux and Windows) swarm mode swarm. I have a VM for Windows Server 1709 and another for Linux.
I’ve created a docker stack with a few Linux and Windows containers defined, all attached to an overlay network.
Ingress routing works - I can publish ports on containers and access them on the external IP of either VM.
Inter-container networking works. I can access containers (and services running on them) from other containers by their internal names, across hosts. (e.g. access Linux containers from Windows and vice versa.)
DNS works. I can ping www.google.com and get an IP address back.
Internet access from Linux containers works. I can ping www.google.com from Linux containers and get ping responses.
Internet access from Windows containers does not work. When I ping www.google.com from a Windows container I get ‘request timed out’, even though I can look up the IP.
Internet access does work from standalone Windows containers.
Expected behavior
Able to access the internet from Windows 1709 containers running in a swarm and attached to an overlay network.
Actual behavior
No internet access from Windows 1709 containers running in swarm mode and attached to an overlay network.
Information
Docker version:
-> docker version
Client:
Version: 18.02.0-ce
API version: 1.34 (downgraded from 1.36)
Go version: go1.9.2
Git commit: fc4de44
Built: Mon Feb 12 08:56:54 2018
OS/Arch: linux/amd64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 17.10.0-ee-preview-3
API version: 1.34 (minimum version 1.24)
Go version: go1.8.4
Git commit: b8571fd
Built: Fri Oct 6 18:01:48 2017
OS/Arch: windows/amd64
Experimental: true
- the output of:
-
DockerDebugInfo.ps1using Powershell on Windows
debuginfo.txt (189.2 KB)
-
docker network ls on the Windows host:
-> docker network ls
NETWORK ID NAME DRIVER SCOPE
jt0sood49kbh ingress overlay swarm
e70385f46f64 nat nat local
2ca023c080a7 none null local
dji43hv3bxib test_overlay overlay swarm
Inspect the test_overlay network:
-> docker network inspect test_overlay
[
{
"Name": "test_overlay",
"Id": "dji43hv3bxib92ldqjxrqiklc",
"Created": "2018-02-19T07:06:43.0971711-08:00",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "10.0.1.0/24",
"Gateway": "10.0.1.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"86bd96a670c8e4ca471e8d858a8f6a0f0eacccc1a06ffa40c7be170f40091a65": {
"Name": "windowsping.1.tszyfp0wk6w452erf2e445i5y",
"EndpointID": "a3bffa491338b805ec756c7d20015ef403a8b0da6198975298a8bf7ed5735ea4",
"MacAddress": "00:15:5d:e5:7c:de",
"IPv4Address": "10.0.1.12/24",
"IPv6Address": ""
},
"test_overlay-sbox": {
"Name": "test_overlay-endpoint",
"EndpointID": "c282173916e6bc6060a66f9b10e590e7ad2155d8f52a70c213ed626cd778db54",
"MacAddress": "00:15:5d:e5:77:25",
"IPv4Address": "10.0.1.3/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "4098",
"com.docker.network.windowsshim.hnsid": "5543adfa-7091-434a-b3a7-3bacaa9ce81f"
},
"Labels": {},
"Peers": [
{
"Name": "41bd519bb433",
"IP": "192.168.84.130"
},
{
"Name": "vagrant-1709-73b05f3333e5",
"IP": "192.168.84.128"
}
]
}
]
Inspect the ‘nat’ network:
-> docker network inspect nat
[
{
"Name": "nat",
"Id": "e70385f46f6416ee615bdc366735ea99be54d977704172719f0abf9268e2f141",
"Created": "2018-02-19T06:22:43.3933081-08:00",
"Scope": "local",
"Driver": "nat",
"EnableIPv6": false,
"IPAM": {
"Driver": "windows",
"Options": null,
"Config": [
{
"Subnet": "172.25.128.0/20",
"Gateway": "172.25.128.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"ed30df8ec3171ac954a3e67545c129fe678fd7d829ce621bcf64a4ae5cfa6229": {
"Name": "windowsping_nonet.1.yo86r9c6vk5yh0iueqyckvl1j",
"EndpointID": "6e422453c4ee008208064571381232daa16023c7d83815632a364049af126397",
"MacAddress": "00:15:5d:22:97:36",
"IPv4Address": "172.25.140.73/16",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.windowsshim.hnsid": "a2cae4d3-0c70-4946-be12-f388cac4e922",
"com.docker.network.windowsshim.networkname": "nat"
},
"Labels": {}
}
]
Steps to reproduce the behavior
- On a linux box, run
docker swarm init - On a Windows Server 1709 box run
docker swarm join - Create an overlay network:
docker network create -d overlay test_overlay - Create a Linux service that runs
ping(note: this works)
-> docker service create --network test_overlay --constraint "node.platform.os == linux" --name linuxping debian ping www.google.com
-> docker service logs linuxping
linuxping.1.lehid1veibtj@gentoo-minimal | 64 bytes from 216.58.204.36: icmp_seq=1805 ttl=127 time=7.934 ms
linuxping.1.lehid1veibtj@gentoo-minimal | 64 bytes from 216.58.204.36: icmp_seq=1806 ttl=127 time=7.922 ms
linuxping.1.lehid1veibtj@gentoo-minimal | 64 bytes from 216.58.204.36: icmp_seq=1807 ttl=127 time=8.146 ms
linuxping.1.lehid1veibtj@gentoo-minimal | 64 bytes from 216.58.204.36: icmp_seq=1808 ttl=127 time=8.085 ms
- Create a Windows service that runs
pingon the overlay network (note: this does not work)
-> docker service create --network test_overlay --constraint "node.platform.os == windows" --name windowsping microsoft/windowsservercore:1709 ping -t www.google.com
-> docker service logs windowsping
windowsping.1.tszyfp0wk6w4@vagrant-1709 | Pinging www.google.com [216.58.211.100] with 32 bytes of data:
windowsping.1.tszyfp0wk6w4@vagrant-1709 | Request timed out.
windowsping.1.tszyfp0wk6w4@vagrant-1709 | Request timed out.
windowsping.1.tszyfp0wk6w4@vagrant-1709 | Request timed out.
windowsping.1.tszyfp0wk6w4@vagrant-1709 | Request timed out.
windowsping.1.tszyfp0wk6w4@vagrant-1709 | Request timed out.
windowsping.1.tszyfp0wk6w4@vagrant-1709 | Request timed out.
windowsping.1.tszyfp0wk6w4@vagrant-1709 | Request timed out.
- Create a Windows service that runs
pingon the default network (note: this works)
-> docker service create --constraint "node.platform.os == windows" --name windowsping_nonet microsoft/windowsservercore:1709 ping -t www.google.com
-> docker service logs windowsping_nonet
windowsping_nonet.1.yo86r9c6vk5y@vagrant-1709 | Reply from 216.58.211.100: bytes=32 time=15ms TTL=127
windowsping_nonet.1.yo86r9c6vk5y@vagrant-1709 | Reply from 216.58.211.100: bytes=32 time=14ms TTL=127
windowsping_nonet.1.yo86r9c6vk5y@vagrant-1709 | Reply from 216.58.211.100: bytes=32 time=15ms TTL=127
windowsping_nonet.1.yo86r9c6vk5y@vagrant-1709 | Reply from 216.58.211.100: bytes=32 time=15ms TTL=127
windowsping_nonet.1.yo86r9c6vk5y@vagrant-1709 | Reply from 216.58.211.100: bytes=32 time=15ms TTL=127
windowsping_nonet.1.yo86r9c6vk5y@vagrant-1709 | Reply from 216.58.211.100: bytes=32 time=15ms TTL=127
Note: For the sake of brevity I haven’t shown inter-container communication here, but I can confirm that Windows and Linux containers can talk to each other over the overlay network and can expose services on the ingress network without issue.
In conclusion
When connected to the overlay network, Windows containers can’t access the internet. When connected to the NAT network, they can.
I think it’s possible that this is due to the fact that Linux uses the docker_gwbridge network for this purpose. Windows doesn’t seem to want to connect containers on an overlay network to the nat network as well, so I’m not sure what to do here.
Help appreciated!