Service is unreachable on Host-IP, localhost works

In my company Kontaktlinsen-Preisvergleich.de - we start playing around with docker, but doesn’t get it to work.

Starting a Nginx should be the easiest thing in the world, but it can’t be reached from outside via the host IP.

Here is what I’ve done:

1.Start nginx:

docker run -d -p 80:80 nginx:1.17.1

2.Check nginx is running:

[root@hblxdev01 ~]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                NAMES
679241b16fc9        nginx:1.17.1        "nginx -g 'daemon of…"   2 seconds ago       Up 1 second         0.0.0.0:80->80/tcp   lucid_goldstine

Looks good! 0.0.0.0:80 means listening on all devices.

3.Reach nginx from localhost (on Host)

[root@hblxdev01 ~]# curl http://localhost
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>...

Yeah, localhost is no problem.

4.Reach nginx from IP (on Host)

[root@hblxdev01 ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 172.17.2.4  netmask 255.255.255.224  broadcast 172.17.2.31
    inet6 fe80::250:56ff:fea9:214c  prefixlen 64  scopeid 0x20<link>
    ether 00:50:56:a9:21:4c  txqueuelen 1000  (Ethernet)

[root@hblxdev01 ~]# curl http://172.17.2.4
curl: (7) Failed connect to 172.17.2.4:80; Connection timed out

That doesn’t work. :sob:

What I have tried and studied so far:

Portforwarding is switched on

[root@hblxdev01 ~]# sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

Firewall is switched off

[root@hblxdev01 ~]# firewall-cmd --state
not running

selinux is disabled

[root@hblxdev01 ~]# getenforce
Disabled

iptables looks good to me

[root@hblxdev01 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy DROP)
target     prot opt source               destination
DOCKER-USER  all  --  anywhere             anywhere
DOCKER-ISOLATION-STAGE-1  all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain DOCKER (3 references)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             172.18.0.2           tcp dpt:http

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target     prot opt source               destination
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere
RETURN     all  --  anywhere             anywhere

Chain DOCKER-ISOLATION-STAGE-2 (3 references)
target     prot opt source               destination
DROP       all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere
RETURN     all  --  anywhere             anywhere

Chain DOCKER-USER (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere

Compared to other installations, I see duplicate entries here, but this may be due to the fact that there are two network interfaces here.

docker is listening on Port 80

[root@hblxdev01 ~]# lsof -i -P -n | grep LISTEN
sshd      3289   root    3u  IPv4  24667      0t0  TCP *:22 (LISTEN)
sshd      3289   root    4u  IPv6  24669      0t0  TCP *:22 (LISTEN)
master    3675   root   14u  IPv4  25039      0t0  TCP 127.0.0.1:25 (LISTEN)
master    3675   root   15u  IPv6  25040      0t0  TCP [::1]:25 (LISTEN)
dockerd   4107   root   24u  IPv6  58803      0t0  TCP *:2377 (LISTEN)
dockerd   4107   root   29u  IPv6  58810      0t0  TCP *:7946 (LISTEN)
docker-pr 7268   root    4u  IPv6  63891      0t0  TCP *:80 (LISTEN)
[root@hblxdev01 ~]# netstat -tulpn | grep LISTEN
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      3289/sshd
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      3675/master
tcp6       0      0 :::2377                 :::*                    LISTEN      4107/dockerd
tcp6       0      0 :::7946                 :::*                    LISTEN      4107/dockerd
tcp6       0      0 :::80                   :::*                    LISTEN      7268/docker-proxy
tcp6       0      0 :::22                   :::*                    LISTEN      3289/sshd
tcp6       0      0 ::1:25                  :::*                    LISTEN      3675/master

This also makes sense, because localhost:80 works, only the IP doesn’t work. I’m not even sure if this is a docker problem, or if there’s an security setting on this machine that I haven’t thought about yet. I am happy about any kind of hint or idea.

Thanks in advance

Marco

More info about the environment

I’m running docker 18.06.3-ce on RHEL 7

$ uname -a
Linux hblxdev01.***********.de 3.10.0-957.21.3.el7.x86_64 #1 SMP Fri Jun 14 02:54:29 EDT 2019 x86_64 x86_64 x86_64 GNU/Linux

$ iptables --version 
iptables v1.4.21

[root@hblxdev01 ~]# docker version
Client:
 Version:           18.06.3-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        d7080c1
 Built:             Wed Feb 20 02:26:51 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
 Version:          18.06.3-ce
 API version:      1.38 (minimum version 1.12)
 Go version:       go1.10.3
 Git commit:       d7080c1
 Built:            Wed Feb 20 02:28:17 2019
 OS/Arch:          linux/amd64
 Experimental:     false

Your LAN’s ip range collides with Docker’s default network ranges.

You might want to read following discussion to get an idea what’s wrong and what needs to be done:

2 Likes

That was exactly the problem. I did what was mentioned in the post and here. I simply added

 "bip": "172.26.0.1/16"

in /etc/docker/daemon.json and restarted the daemon. Now everything works as expected!

Thank you very much @meyay !!!

1 Like