Docker Service endpoints is not accessible in Windows Server 2016 container swarm mode

Docker Desktop for Windows
1

I am deploying docker stack on Windows server 2016 containers swarm mode with below mentioned stack.yml. Its getting deploying properly, however (1) service endpoint stack_web is not accessible as @published uri http://hostcomputer:80 and (2) both db & web service endpoint ips are not reachable on stack_net network

    version: "3"
    services:
        db:
            image: mongo
            networks:
                    net:
                        aliases:
                            -  mymongodb
        web:
            image: microsoft/iis
            ports: 
                - 80:80
            networks:
                - net
    networks:
        net:

Deploying command used :
PS > docker stack up -c mongoiis.yml app1stack
Creating network app1stack_net
Creating service app1stack_db
Creating service app1stack_web

PS > docker stack services app1stack
ID NAME MODE REPLICAS IMAGE PORTS
c0bdvuccpde9 app1stack_db replicated 1/1 mongo:latest
ngv38tj5mpof app1stack_web replicated 1/1 microsoft/iis:latest *:80->80/tcp

PS > docker stack ps app1stack
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
xylm7wgweyjl app1stack_db.1 mongo:latest democonsul Running Running 4 minutes ago
qpmtx7knk6m5 app1stack_web.1 microsoft/iis:latest democonsul Running Running 7 minutes ago

–

2

adding more diagnostic info:

PS > docker stack services app1stack
ID NAME MODE REPLICAS IMAGE PORTS
c0bdvuccpde9 app1stack_db replicated 1/1 mongo:latest
ngv38tj5mpof app1stack_web replicated 1/1 microsoft/iis:latest *:80->80/tcp

PS > ping app1stack_web
Ping request could not find host app1stack_web. Please check the name and try again.

PS > iwr http://localhost -UseBasicParsing
iwr : Unable to connect to the remote server
At line:1 char:1

  • iwr http://localhost -UseBasicParsing
  •   + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
  + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

PS > docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b50429f850cc mongo:latest “mongod” 10 minutes ago Up 9 minutes 27017/tcp app1stack_db.1.xylm7wgweyjlv5vwgl3ukxv7q
b5edbc4fc432 microsoft/iis:latest “C:\ServiceMonitor…” 12 minutes ago Up 12 minutes 80/tcp app1stack_web.1.qpmtx7knk6m5n9msgo9jex8jv
8dfd1e2d1455 docker/ucp-agent-win:2.2.3 “/bin/ucp-agent.ex…” 33 hours ago Up 2 hours 0.0.0.0:12376->2376/tcp ucp-proxy

PS > docker exec b5edbc4fc432 ping app1stack_db

Pinging app1stack_db [10.0.0.2] with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 10.0.0.2:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

PS > docker exec b5edbc4fc432 ping b50429f850cc

Pinging b50429f850cc [10.0.0.3] with 32 bytes of data:
Reply from 10.0.0.3: bytes=32 time<1ms TTL=128
Reply from 10.0.0.3: bytes=32 time<1ms TTL=128
Reply from 10.0.0.3: bytes=32 time<1ms TTL=128
Reply from 10.0.0.3: bytes=32 time<1ms TTL=128

Ping statistics for 10.0.0.3:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms

PS > docker service inspect app1stack_web
[
{
“ID”: “ngv38tj5mpofy9tcwwlqlqdtn”,
“Version”: {
“Index”: 805
},
“CreatedAt”: “2017-10-09T13:37:50.2881996Z”,
“UpdatedAt”: “2017-10-09T13:39:04.6215664Z”,
“Spec”: {
“Name”: “app1stack_web”,
“Labels”: {
“com.docker.stack.image”: “microsoft/iis”,
“com.docker.stack.namespace”: “app1stack”
},
“TaskTemplate”: {
“ContainerSpec”: {
“Image”: “microsoft/iis:latest@sha256:868bca7e89e1743792e15f78edb5a73070ef44eae6807dc3f05f9b94c23943d5”,
“Labels”: {
“com.docker.stack.namespace”: “app1stack”
},
“Privileges”: {
“CredentialSpec”: null,
“SELinuxContext”: null
},
“StopGracePeriod”: 10000000000,
“DNSConfig”: {}
},
“Resources”: {},
“RestartPolicy”: {
“Condition”: “any”,
“Delay”: 5000000000,
“MaxAttempts”: 0
},
“Placement”: {
“Platforms”: [
{
“Architecture”: “amd64”,
“OS”: “windows”
}
]
},
“Networks”: [
{
“Target”: “053w1jawq9ic5apxwcqy1rl2b”,
“Aliases”: [
“web”
]
}
],
“ForceUpdate”: 0,
“Runtime”: “container”
},
“Mode”: {
“Replicated”: {
“Replicas”: 1
}
},
“UpdateConfig”: {
“Parallelism”: 1,
“FailureAction”: “pause”,
“Monitor”: 5000000000,
“MaxFailureRatio”: 0,
“Order”: “stop-first”
},
“RollbackConfig”: {
“Parallelism”: 1,
“FailureAction”: “pause”,
“Monitor”: 5000000000,
“MaxFailureRatio”: 0,
“Order”: “stop-first”
},
“EndpointSpec”: {
“Mode”: “vip”,
“Ports”: [
{
“Protocol”: “tcp”,
“TargetPort”: 80,
“PublishedPort”: 80,
“PublishMode”: “ingress”
}
]
}
},
“Endpoint”: {
“Spec”: {
“Mode”: “vip”,
“Ports”: [
{
“Protocol”: “tcp”,
“TargetPort”: 80,
“PublishedPort”: 80,
“PublishMode”: “ingress”
}
]
},
“Ports”: [
{
“Protocol”: “tcp”,
“TargetPort”: 80,
“PublishedPort”: 80,
“PublishMode”: “ingress”
}
],
“VirtualIPs”: [
{
“NetworkID”: “pytj7xgaf7zig7ugx0f7qqly3”,
“Addr”: “10.255.0.4/16”
},
{
“NetworkID”: “053w1jawq9ic5apxwcqy1rl2b”,
“Addr”: “10.0.0.4/24”
}
]
}
}
]

PS > docker service inspect app1stack_db
[
{
“ID”: “c0bdvuccpde9dmmjxv0b2iaqb”,
“Version”: {
“Index”: 805
},
“CreatedAt”: “2017-10-09T13:37:47.6728508Z”,
“UpdatedAt”: “2017-10-09T13:39:04.6215664Z”,
“Spec”: {
“Name”: “app1stack_db”,
“Labels”: {
“com.docker.stack.image”: “mongo”,
“com.docker.stack.namespace”: “app1stack”
},
“TaskTemplate”: {
“ContainerSpec”: {
“Image”: “mongo:latest@sha256:bb962ca3964c70069e652e814f7a495db19643ecb5bb62d799f621a68f4da90c”,
“Labels”: {
“com.docker.stack.namespace”: “app1stack”
},
“Privileges”: {
“CredentialSpec”: null,
“SELinuxContext”: null
},
“StopGracePeriod”: 10000000000,
“DNSConfig”: {}
},
“Resources”: {},
“RestartPolicy”: {
“Condition”: “any”,
“Delay”: 5000000000,
“MaxAttempts”: 0
},
“Placement”: {
“Platforms”: [
{
“Architecture”: “amd64”,
“OS”: “linux”
},
{
“Architecture”: “amd64”,
“OS”: “windows”
}
]
},
“Networks”: [
{
“Target”: “053w1jawq9ic5apxwcqy1rl2b”,
“Aliases”: [
“mymongodb”,
“db”
]
}
],
“ForceUpdate”: 0,
“Runtime”: “container”
},
“Mode”: {
“Replicated”: {
“Replicas”: 1
}
},
“UpdateConfig”: {
“Parallelism”: 1,
“FailureAction”: “pause”,
“Monitor”: 5000000000,
“MaxFailureRatio”: 0,
“Order”: “stop-first”
},
“RollbackConfig”: {
“Parallelism”: 1,
“FailureAction”: “pause”,
“Monitor”: 5000000000,
“MaxFailureRatio”: 0,
“Order”: “stop-first”
},
“EndpointSpec”: {
“Mode”: “vip”
}
},
“Endpoint”: {
“Spec”: {
“Mode”: “vip”
},
“VirtualIPs”: [
{
“NetworkID”: “053w1jawq9ic5apxwcqy1rl2b”,
“Addr”: “10.0.0.2/24”
}
]
}
}
]

PS >

3

From the perspective of the web container, the DB containers DNS name is just db and not appstack1_db. The stack name is pretended to the service name (with a trailing ordinal) to form the container name but that name doesn’t get added to DNS within the docker network created in the stack file. You should be able to successfully run:

docker exec b5edbc4fc432 ping db

Not sure though why you can’t reach the webserver from the host.

4

Thank you Dannc !! Both db and appstack_db DNS names are resolving and associated IP is same 10.0.0.2 which is not reachable when ping from web service containers or for db client connection request. 10.0.0.2 is the IP assigned as VIP in PS> docker inspect appstack1_db .

5

Interesting. I looked over your “service inspect” commands (now that I’m on my laptop and not on my phone) and clearly they both show as being attached to the same docker network. When running “docker network inspect app1stack_net” do both of the containers also show as being attached to that network?

I find it odd for sure that you can ping the container id directly but not the name/ip associated with the service, which points to something mucked up in the docker network routing as opposed to something in the containers themselves.

Note: I just went looking and it appears that Docker Swarm Mode isn’t supported for Window Server 2016 (see https://github.com/moby/moby/issues/27612) but is for Window 10 Creators Update. It looks like though the MS’s doc https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/swarm-mode contradicts this with some info about how to setup the Windows Server 2016 node for this. Have you tried that yet?

6

Thank you !!

MS document @ https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/swarm-mode helps to get better understanding issue #2 i.e --endpoint-mode VIP is not supported in Windows Server. So i changed my yml file with with endpoint-mode: dnsrr, Now my web container can ping db services. However I am unable to access service on published port http://dockerwindowshost:publishedport

version: "3.3"
services:
  db:
    image: mongo:3.4.9-windowsservercore
    networks:
       net:
          aliases:
            - mymongodb
        deploy:
          endpoint_mode: dnsrr
  web:
    image: microsoft/iis
    ports:
      - target: 80
        published: 80
        protocol: tcp
        mode: host
    networks:
      - net
    deploy:
      endpoint_mode: dnsrr
    depends_on:
        - db
networks:
  net:
7

Not really sure what’s going on with external access to the web container. Any chance that it’s being blocked via firewall or some other policy? If you exec into the web container itself, can you connect to port 80 there? Maybe double check with netstat that it is even listening…

8

Thank you for your reply!!

Yes… firewall is good! if I run them as container I can access application container on http://containerhostname/ip:80 and also in any mapped port on http://windowsserverhostname:port (if i run with -p 80:port). I see same all policies are allowed to access ports 80.

9

Hmmm. Looking at this that should work. Not 100% sure why it wouldn’t.

As an experiment, I wonder if you could add a second network to your configuration like this:

net2:
       driver: bridge

and then add it to your web service. That way you’re taking the overlay network as a source of potential port mapping issues out of the equation.

10

Driver:bridge is not supported in windows server, equivalent is driver:transparent. However driver:transparent is not supported for swarm mode services.

PS>docker info
Containers: 5
Running: 1
Paused: 0
Stopped: 4
Images: 8
Server Version: 17.06.2-ee-4
Storage Driver: windowsfilter
Windows:
Logging Driver: json-file
Plugins:
Volume: local
Network: l2bridge l2tunnel nat null overlay transparent
Log: awslogs etwlogs fluentd json-file logentries splunk syslog
NodeID: tt9s6joyfmdbqrn6j4d1a6z4q
Is Manager: false
Node Address: 10.100.0.7
Manager Addresses:
10.100.0.6:2377
Default Isolation: process
Kernel Version: 10.0 14393 (14393.1770.amd64fre.rs1_release.170917-1700)
Operating System: Windows Server 2016 Datacenter
OSType: windows
Architecture: x86_64
CPUs: 2
Total Memory: 8GiB
Name: consulcr
ID: CD6F:UDGC:LDX4:SYCE:6BDI:OR63:2O5A:Z33Z:UOOG:22JK:WSZ4:PWJV
Docker Root Dir: C:\ProgramData\docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

PS>docker network create -d bridge bridgenet
Error response from daemon: plugin not found