Docker Community Forums

Share and learn in the Docker community.

Containers can't ping over overlay network on Windows


(Gh2k) #1

I’m not sure what’s happening here:

I have a development swarm of 2 Linux hosts and 2 Windows Server hosts. One of the Linux hosts is the swarm manager - the rest are workers.

I’ve created a service replicated across my Linux nodes, and another replicated across my Windows ones. There is one container running on each host, as expected.

Let’s call the Linux service ‘serviceA’ and the Windows service ‘serviceB’

The services are connected to an overlay network. The network is visible on all hosts.

I’ve created the services with ‘dnsrr’ as the endpoint mode, and ‘host’ as the publish mode.

When the environment is running, serviceA can’t see serviceB.

Further testing by executing a shell inside the running containers shows that Windows is unable to communicate over the overlay network, when I try to access another container. For example, when I’ve got a shell open on a serviceA container, the dns entry I get back for serviceB alternates between the internal container IPs (10.0.0.x) but I can neither ping or connect to the service on this address. I can access the 10.0.0.x address for the other serviceA container just fine.

From Windows, I’m unable to access either 10.0.0.x container even though I get these addresses back from DNS. This is both via ping and connecting to the service directly.

I’m aware that mesh routing does not work in Windows Server atm. Does this mean that I shouldn’t be able to connect to the 10.x addresses at all? - I can access the services via the host’s external IP without issue, but that’s not the address that I get back from DNS.

Here’s what one of my services looks like, when inspected:

{
    "ID": "ov045m3616y68abt56530fvko",
    "Version": {
        "Index": 1565
    },
    "CreatedAt": "2017-10-23T14:32:35.572104962Z",
    "UpdatedAt": "2017-10-23T16:02:19.815896425Z",
    "Spec": {
        "Name": "serviceB",
        "Labels": {
            "com.docker.stack.image": "myimage",
            "com.docker.stack.namespace": "myns"
        },
        "TaskTemplate": {
            "ContainerSpec": {
                "Image": "myimage@blah",
                "Labels": {
                    "com.docker.stack.namespace": "myns"
                },
                "Privileges": {
                    "CredentialSpec": null,
                    "SELinuxContext": null
                },
                "StopGracePeriod": 10000000000,
                "DNSConfig": {}
            },
            "Resources": {},
            "RestartPolicy": {
                "Condition": "any",
                "Delay": 5000000000,
                "MaxAttempts": 0
            },
            "Placement": {
                "Constraints": [
                    "node.platform.os == windows"
                ],
                "Platforms": [
                    {
                        "Architecture": "amd64",
                        "OS": "windows"
                    }
                ]
            },
            "Networks": [
                {
                    "Target": "5fukysm14msotumdv261bormt",
                    "Aliases": [
                        "serviceB"
                    ]
                }
            ],
            "ForceUpdate": 0,
            "Runtime": "container"
        },
        "Mode": {
            "Replicated": {
                "Replicas": 3
            }
        },
        "UpdateConfig": {
            "Parallelism": 1,
            "FailureAction": "pause",
            "Monitor": 5000000000,
            "MaxFailureRatio": 0,
            "Order": "stop-first"
        },
        "RollbackConfig": {
            "Parallelism": 1,
            "FailureAction": "pause",
            "Monitor": 5000000000,
            "MaxFailureRatio": 0,
            "Order": "stop-first"
        },
        "EndpointSpec": {
            "Mode": "dnsrr",
            "Ports": [
                {
                    "Protocol": "tcp",
                    "TargetPort": 1979,
                    "PublishedPort": 1979,
                    "PublishMode": "host"
                }
            ]
        }
    },
    "PreviousSpec": {
        "Name": "myns_serviceB",
        "Labels": {
            "com.docker.stack.image": "myimage",
            "com.docker.stack.namespace": "myns"
        },
        "TaskTemplate": {
            "ContainerSpec": {
                "Image": "myimage@blah",
                "Labels": {
                    "com.docker.stack.namespace": "myns"
                },
                "Privileges": {
                    "CredentialSpec": null,
                    "SELinuxContext": null
                }
            },
            "Resources": {},
            "RestartPolicy": {
                "Condition": "any",
                "MaxAttempts": 0
            },
            "Placement": {
                "Constraints": [
                    "node.platform.os == windows"
                ],
                "Platforms": [
                    {
                        "Architecture": "amd64",
                        "OS": "windows"
                    }
                ]
            },
            "Networks": [
                {
                    "Target": "5fukysm14msotumdv261bormt",
                    "Aliases": [
                        "serviceB"
                    ]
                }
            ],
            "ForceUpdate": 0,
            "Runtime": "container"
        },
        "Mode": {
            "Replicated": {
                "Replicas": 2
            }
        },
        "EndpointSpec": {
            "Mode": "dnsrr",
            "Ports": [
                {
                    "Protocol": "tcp",
                    "TargetPort": 1979,
                    "PublishedPort": 1979,
                    "PublishMode": "host"
                }
            ]
        }
    },
    "Endpoint": {
        "Spec": {
            "Mode": "dnsrr",
            "Ports": [
                {
                    "Protocol": "tcp",
                    "TargetPort": 1979,
                    "PublishedPort": 1979,
                    "PublishMode": "host"
                }
            ]
        },
        "Ports": [
            {
                "Protocol": "tcp",
                "TargetPort": 1979,
                "PublishedPort": 1979,
                "PublishMode": "host"
            }
        ]
    }
}

My network looks like this:

{
    "Name": "myns_mynetwork",
    "Id": "5fukysm14msotumdv261bormt",
    "Created": "2017-10-23T14:32:39.125171182Z",
    "Scope": "swarm",
    "Driver": "overlay",
    "EnableIPv6": false,
    "IPAM": {
        "Driver": "default",
        "Options": null,
        "Config": [
            {
                "Subnet": "10.0.0.0/24",
                "Gateway": "10.0.0.1"
            }
        ]
    },
    "Internal": false,
    "Attachable": false,
    "Ingress": false,
    "ConfigFrom": {
        "Network": ""
    },
    "ConfigOnly": false,
    "Containers": {
        "0e2fc2c472cf92ea6ef20788413b86d0e9dae2f6e9ca6549674aaeb3970af759": {
            "Name": "mylinuxcontainer",
            "EndpointID": "77b010672c7a061c5258e2e7ad8e3b25535e06f9f206cbc5f227852600e6af05",
            "MacAddress": "02:42:0a:00:00:03",
            "IPv4Address": "10.0.0.3/24",
            "IPv6Address": ""
        }
    },
    "Options": {
        "com.docker.network.driver.overlay.vxlanid_list": "4097"
    },
    "Labels": {
        "com.docker.stack.namespace": "myns"
    },
    "Peers": [
        {
            "Name": "linux-blah",
            "IP": "192.168.34.10"
        },
        {
            "Name": "windows-blah",
            "IP": "192.168.34.12"
        }
    ]
}

There are only two peers listed above because I’m running a smaller environment for testing this further. Note that when running ‘network inspect’ as above, I only see the Linux container on the Linux host and the Windows container on the Windows host. I’m not sure if this is what’s supposed to happen.

I tried disabling firewalls on both the Linux and Windows hosts, but that didn’t help.

At this point, I’m out of ideas and need some help please. :slight_smile:


(Dann Church) #2

If you’re running Window Server 2016, the Overlay networking isn’t 100% supported yet. I found this from Microsoft and it has some instructions that was useful to at least one other member of the forums.

Also, apparently this should work on the latest Windows 10 creator update.

Good luck!


(Lanwin) #3

I am faced with the same problem as yours. Did you found a solution yet?


(Gh2k) #4

No, unfortunately not. I’ve been trying to use Windows Server 1709 which in theory supports overlay networking without dnsrr and host ingress mode, but so far I’ve not managed to get that working either. See https://github.com/docker/for-win/issues/1366