Docker Pets does not run

General
1

Hello guys,

I’m trying to setup my demo site but there’s an issue after the docker pets deployment: docker1.example.com:3000 refused to connect.

Environment:
OS: SUSE Linux Enterprise Server 12 SP2
Docker EE: 17.03.2-ee-4
UCP: 2.1.4
Networking:
docker1.example.com 192.168.188.25, UCP leader
docker2.example.com 192.168.188.26, worker
docker3.example.com 192.168.188.27, worker

Steps to duplicate:

  1. Installation docker EE on three nodes
  2. Install UCP using the following command
    docker run --rm -it --name ucp -v /var/run/docker.sock:/var/run/docker.sock docker/ucp:2.1.4 install --debug --admin-username admin --admin-password 1qaz2wsx --san docker1.example.com --host-address 192.168.188.25
  3. Deploy a service using compose.yml definition
    (Following https://github.com/docker/dcus-hol-2017/tree/master/docker-enterprise#task1)
    Name: pets
    Deploy as: services
    Definition:
    version: '3.1' services: web: image: chrch/docker-pets:1.0 ports: - 5000 deploy: replicas: 2 healthcheck: interval: 10s timeout: 2s retries: 3
  4. Click on Services -> Details -> Published Ports, you will see:
    5000/tcp Ingress: 30000 Ingress: docker1.example.com:30000
  5. Open your browser, browse to docker1.example.com:30000, issue here:
    This site can’t be reached. docker1.example.com refused to connect.

P.S:.

  1. I tried to use shell and login on the container, it’s running well with port 5000 listening.

  2. Nothing error during the docker/ucp installation and the docker pets service deployment.

  3. I’ve checked the DNS, everything is working fine.

  4. I tried to telnet docker1.example.com 30000 from my client (A windows box), it shows me connection refused. I guess the iptables rules doesn’t work on docker1 host because it doesn’t show anything port 3000 to redirect… not sure so I list all the rules running here:

    docker1:~ # iptables-save

    Generated by iptables-save v1.4.21 on Thu Jul 6 09:42:18 2017

    *mangle
    :PREROUTING ACCEPT [348100:205179157]
    :INPUT ACCEPT [316999:169421923]
    :FORWARD ACCEPT [30954:35731810]
    :OUTPUT ACCEPT [297285:118983815]
    :POSTROUTING ACCEPT [328239:154715625]
    COMMIT

    Completed on Thu Jul 6 09:42:18 2017

    Generated by iptables-save v1.4.21 on Thu Jul 6 09:42:18 2017

    *nat
    :PREROUTING ACCEPT [7230:489898]
    :INPUT ACCEPT [7009:460034]
    :OUTPUT ACCEPT [6631:398884]
    :POSTROUTING ACCEPT [6667:400836]
    :DOCKER - [0:0]
    -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
    -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
    -A POSTROUTING -s 172.18.0.0/16 ! -o docker_gwbridge -j MASQUERADE
    -A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
    -A POSTROUTING -s 172.17.0.6/32 -d 172.17.0.6/32 -p tcp -m tcp --dport 2376 -j MASQUERADE
    -A POSTROUTING -s 172.17.0.7/32 -d 172.17.0.7/32 -p tcp -m tcp --dport 12381 -j MASQUERADE
    -A POSTROUTING -s 172.17.0.8/32 -d 172.17.0.8/32 -p tcp -m tcp --dport 12382 -j MASQUERADE
    -A POSTROUTING -s 172.17.0.9/32 -d 172.17.0.9/32 -p tcp -m tcp --dport 12380 -j MASQUERADE
    -A POSTROUTING -s 172.17.0.9/32 -d 172.17.0.9/32 -p tcp -m tcp --dport 2379 -j MASQUERADE
    -A POSTROUTING -s 172.17.0.10/32 -d 172.17.0.10/32 -p tcp -m tcp --dport 12384 -j MASQUERADE
    -A POSTROUTING -s 172.17.0.10/32 -d 172.17.0.10/32 -p tcp -m tcp --dport 12383 -j MASQUERADE
    -A POSTROUTING -s 172.17.0.11/32 -d 172.17.0.11/32 -p tcp -m tcp --dport 4443 -j MASQUERADE
    -A POSTROUTING -s 172.17.0.12/32 -d 172.17.0.12/32 -p tcp -m tcp --dport 2375 -j MASQUERADE
    -A POSTROUTING -s 172.17.0.13/32 -d 172.17.0.13/32 -p tcp -m tcp --dport 12387 -j MASQUERADE
    -A POSTROUTING -s 172.17.0.14/32 -d 172.17.0.14/32 -p tcp -m tcp --dport 4443 -j MASQUERADE
    -A POSTROUTING -s 172.17.0.15/32 -d 172.17.0.15/32 -p tcp -m tcp --dport 8080 -j MASQUERADE
    -A DOCKER -i docker_gwbridge -j RETURN
    -A DOCKER -i docker0 -j RETURN
    -A DOCKER ! -i docker0 -p tcp -m tcp --dport 12376 -j DNAT --to-destination 172.17.0.6:2376
    -A DOCKER ! -i docker0 -p tcp -m tcp --dport 12381 -j DNAT --to-destination 172.17.0.7:12381
    -A DOCKER ! -i docker0 -p tcp -m tcp --dport 12382 -j DNAT --to-destination 172.17.0.8:12382
    -A DOCKER ! -i docker0 -p tcp -m tcp --dport 12380 -j DNAT --to-destination 172.17.0.9:12380
    -A DOCKER ! -i docker0 -p tcp -m tcp --dport 12379 -j DNAT --to-destination 172.17.0.9:2379
    -A DOCKER ! -i docker0 -p tcp -m tcp --dport 12384 -j DNAT --to-destination 172.17.0.10:12384
    -A DOCKER ! -i docker0 -p tcp -m tcp --dport 12383 -j DNAT --to-destination 172.17.0.10:12383
    -A DOCKER ! -i docker0 -p tcp -m tcp --dport 12385 -j DNAT --to-destination 172.17.0.11:4443
    -A DOCKER ! -i docker0 -p tcp -m tcp --dport 2376 -j DNAT --to-destination 172.17.0.12:2375
    -A DOCKER ! -i docker0 -p tcp -m tcp --dport 12387 -j DNAT --to-destination 172.17.0.13:12387
    -A DOCKER ! -i docker0 -p tcp -m tcp --dport 12386 -j DNAT --to-destination 172.17.0.14:4443
    -A DOCKER ! -i docker0 -p tcp -m tcp --dport 443 -j DNAT --to-destination 172.17.0.15:8080
    COMMIT

    Completed on Thu Jul 6 09:42:18 2017

    Generated by iptables-save v1.4.21 on Thu Jul 6 09:42:18 2017

    *filter
    :INPUT ACCEPT [314724:168902096]
    :FORWARD DROP [0:0]
    :OUTPUT ACCEPT [295067:118442221]
    :DOCKER - [0:0]
    :DOCKER-ISOLATION - [0:0]
    -A FORWARD -j DOCKER-ISOLATION
    -A FORWARD -o docker_gwbridge -j DOCKER
    -A FORWARD -o docker_gwbridge -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
    -A FORWARD -i docker_gwbridge ! -o docker_gwbridge -j ACCEPT
    -A FORWARD -o docker0 -j DOCKER
    -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
    -A FORWARD -i docker0 ! -o docker0 -j ACCEPT
    -A FORWARD -i docker0 -o docker0 -j ACCEPT
    -A FORWARD -i docker_gwbridge -o docker_gwbridge -j DROP
    -A DOCKER -d 172.17.0.6/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 2376 -j ACCEPT
    -A DOCKER -d 172.17.0.7/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 12381 -j ACCEPT
    -A DOCKER -d 172.17.0.8/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 12382 -j ACCEPT
    -A DOCKER -d 172.17.0.9/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 12380 -j ACCEPT
    -A DOCKER -d 172.17.0.9/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 2379 -j ACCEPT
    -A DOCKER -d 172.17.0.10/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 12384 -j ACCEPT
    -A DOCKER -d 172.17.0.10/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 12383 -j ACCEPT
    -A DOCKER -d 172.17.0.11/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 4443 -j ACCEPT
    -A DOCKER -d 172.17.0.12/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 2375 -j ACCEPT
    -A DOCKER -d 172.17.0.13/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 12387 -j ACCEPT
    -A DOCKER -d 172.17.0.14/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 4443 -j ACCEPT
    -A DOCKER -d 172.17.0.15/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8080 -j ACCEPT
    -A DOCKER-ISOLATION -i docker0 -o docker_gwbridge -j DROP
    -A DOCKER-ISOLATION -i docker_gwbridge -o docker0 -j DROP
    -A DOCKER-ISOLATION -j RETURN
    COMMIT

    Completed on Thu Jul 6 09:42:18 2017

    docker1:~ #

2

Hello All,

Problem solved with a package ‘cluster-network-kmp-default’ installed. This package provides kernel module called ‘ip_vs’ (Linux IPVS).

By default, SLES12 kernel doesn’t include IP Virtual Server kernel module. This will cause UCP failed to load ‘ip_vs’ kernel module. There is no issue when you’re running Docker EE on a single node deployment. If you want to run swarm cluster with UCP, you need to add-on a product extension from SUSE which is called ‘SUSE Enterprise Server 12 with High Availability Extension (HAE)’ then you will have a RPM package called ‘cluster-network-kmp-default’.

Regards,
Leo Liu