I’m not sure what’s happening here:
I have a development swarm of 2 Linux hosts and 2 Windows Server hosts. One of the Linux hosts is the swarm manager - the rest are workers.
I’ve created a service replicated across my Linux nodes, and another replicated across my Windows ones. There is one container running on each host, as expected.
Let’s call the Linux service ‘serviceA’ and the Windows service ‘serviceB’
The services are connected to an overlay network. The network is visible on all hosts.
I’ve created the services with ‘dnsrr’ as the endpoint mode, and ‘host’ as the publish mode.
When the environment is running, serviceA can’t see serviceB.
Further testing by executing a shell inside the running containers shows that Windows is unable to communicate over the overlay network, when I try to access another container. For example, when I’ve got a shell open on a serviceA container, the dns entry I get back for serviceB alternates between the internal container IPs (10.0.0.x) but I can neither ping or connect to the service on this address. I can access the 10.0.0.x address for the other serviceA container just fine.
From Windows, I’m unable to access either 10.0.0.x container even though I get these addresses back from DNS. This is both via ping and connecting to the service directly.
I’m aware that mesh routing does not work in Windows Server atm. Does this mean that I shouldn’t be able to connect to the 10.x addresses at all? - I can access the services via the host’s external IP without issue, but that’s not the address that I get back from DNS.
Here’s what one of my services looks like, when inspected:
{
"ID": "ov045m3616y68abt56530fvko",
"Version": {
"Index": 1565
},
"CreatedAt": "2017-10-23T14:32:35.572104962Z",
"UpdatedAt": "2017-10-23T16:02:19.815896425Z",
"Spec": {
"Name": "serviceB",
"Labels": {
"com.docker.stack.image": "myimage",
"com.docker.stack.namespace": "myns"
},
"TaskTemplate": {
"ContainerSpec": {
"Image": "myimage@blah",
"Labels": {
"com.docker.stack.namespace": "myns"
},
"Privileges": {
"CredentialSpec": null,
"SELinuxContext": null
},
"StopGracePeriod": 10000000000,
"DNSConfig": {}
},
"Resources": {},
"RestartPolicy": {
"Condition": "any",
"Delay": 5000000000,
"MaxAttempts": 0
},
"Placement": {
"Constraints": [
"node.platform.os == windows"
],
"Platforms": [
{
"Architecture": "amd64",
"OS": "windows"
}
]
},
"Networks": [
{
"Target": "5fukysm14msotumdv261bormt",
"Aliases": [
"serviceB"
]
}
],
"ForceUpdate": 0,
"Runtime": "container"
},
"Mode": {
"Replicated": {
"Replicas": 3
}
},
"UpdateConfig": {
"Parallelism": 1,
"FailureAction": "pause",
"Monitor": 5000000000,
"MaxFailureRatio": 0,
"Order": "stop-first"
},
"RollbackConfig": {
"Parallelism": 1,
"FailureAction": "pause",
"Monitor": 5000000000,
"MaxFailureRatio": 0,
"Order": "stop-first"
},
"EndpointSpec": {
"Mode": "dnsrr",
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 1979,
"PublishedPort": 1979,
"PublishMode": "host"
}
]
}
},
"PreviousSpec": {
"Name": "myns_serviceB",
"Labels": {
"com.docker.stack.image": "myimage",
"com.docker.stack.namespace": "myns"
},
"TaskTemplate": {
"ContainerSpec": {
"Image": "myimage@blah",
"Labels": {
"com.docker.stack.namespace": "myns"
},
"Privileges": {
"CredentialSpec": null,
"SELinuxContext": null
}
},
"Resources": {},
"RestartPolicy": {
"Condition": "any",
"MaxAttempts": 0
},
"Placement": {
"Constraints": [
"node.platform.os == windows"
],
"Platforms": [
{
"Architecture": "amd64",
"OS": "windows"
}
]
},
"Networks": [
{
"Target": "5fukysm14msotumdv261bormt",
"Aliases": [
"serviceB"
]
}
],
"ForceUpdate": 0,
"Runtime": "container"
},
"Mode": {
"Replicated": {
"Replicas": 2
}
},
"EndpointSpec": {
"Mode": "dnsrr",
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 1979,
"PublishedPort": 1979,
"PublishMode": "host"
}
]
}
},
"Endpoint": {
"Spec": {
"Mode": "dnsrr",
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 1979,
"PublishedPort": 1979,
"PublishMode": "host"
}
]
},
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 1979,
"PublishedPort": 1979,
"PublishMode": "host"
}
]
}
}
My network looks like this:
{
"Name": "myns_mynetwork",
"Id": "5fukysm14msotumdv261bormt",
"Created": "2017-10-23T14:32:39.125171182Z",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "10.0.0.0/24",
"Gateway": "10.0.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"0e2fc2c472cf92ea6ef20788413b86d0e9dae2f6e9ca6549674aaeb3970af759": {
"Name": "mylinuxcontainer",
"EndpointID": "77b010672c7a061c5258e2e7ad8e3b25535e06f9f206cbc5f227852600e6af05",
"MacAddress": "02:42:0a:00:00:03",
"IPv4Address": "10.0.0.3/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "4097"
},
"Labels": {
"com.docker.stack.namespace": "myns"
},
"Peers": [
{
"Name": "linux-blah",
"IP": "192.168.34.10"
},
{
"Name": "windows-blah",
"IP": "192.168.34.12"
}
]
}
There are only two peers listed above because I’m running a smaller environment for testing this further. Note that when running ‘network inspect’ as above, I only see the Linux container on the Linux host and the Windows container on the Windows host. I’m not sure if this is what’s supposed to happen.
I tried disabling firewalls on both the Linux and Windows hosts, but that didn’t help.
At this point, I’m out of ideas and need some help please. 