Container doesn't bind on ip:port

General
1

Hi,

maybe it’s to late in the evening, but I’m not able to setup a network binding/ip with port, e.g.:

$ docker run -d --rm --name=portainer \
-v data:/data -v /var/run/docker.sock:/var/run/docker.sock \
--net=dmz_net --ip=192.168.90.20 -p 192.168.90.20:8008:9000 \
portainer/portainer

It shall setup on 192.168.90.20 (dmz_net) port 8008 (or even 80, which doesn’t work too).

$ docker inspect portainer
[
  ...
            "NetworkMode": "dmz_net",
            "PortBindings": {
                "9000/tcp": [
                    {
                        "HostIp": "192.168.90.20",
                        "HostPort": "8008"
                    }
                ]
            },
....
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "e039d28d590eb947e5892e3bc73454229ccb8d52e6656b5cdf98a12d1c90a150",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/e039d28d590e",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "dmz_net": {
                    "IPAMConfig": {
                        "IPv4Address": "192.168.90.20"
                    },
                    "Links": null,
                    "Aliases": [
                        "e15beb2c36e5"
                    ],
                    "NetworkID": "01d26c8ac1983029a087a6c509036180a432f9894ff78a9860d2a04aa3a9693a",
                    "EndpointID": "4d34ce23ee474db9a2eb6e50adf98cc9bcc616aa91f08f43b6fc11fbc0239a89",
                    "Gateway": "192.168.90.1",
                    "IPAddress": "192.168.90.20",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:c0:a8:5a:14",
                    "DriverOpts": null
                }
            }
        }
    }
]

also, netstat doesn’t show anything:

$ sudo netstat -tulpen
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode      PID/Program name   
tcp        0      0 127.0.0.1:40231         0.0.0.0:*               LISTEN      0          21137      398/containerd      
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      98         20051      369/systemd-resolve 
tcp        0      0 :::22                   :::*                    LISTEN      0          2755       1/systemd           
udp        0      0 127.0.0.53:53           0.0.0.0:*                           98         20050      369/systemd-resolve

but probably I miss something here (I’m new to vlan networking). Nevertheless, I can connect to the default port successfully:

$ curl -I 192.168.90.20:9000
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=31536000
Content-Length: 23032
Content-Type: text/html; charset=utf-8
Last-Modified: Thu, 19 Mar 2020 22:46:17 GMT
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block
Date: Tue, 19 May 2020 19:16:17 GMT

but not from port intended:

$ curl -I 192.168.90.20:8008
curl: (7) Failed to connect to 192.168.90.20 port 8008: Connection refused

The docker network is VLAN:

$ docker inspect dmz_net
[
    {
        "Name": "dmz_net",
        "Id": "01d26c8ac1983029a087a6c509036180a432f9894ff78a9860d2a04aa3a9693a",
        "Created": "2020-05-19T20:10:05.768577941+02:00",
        "Scope": "local",
        "Driver": "macvlan",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "192.168.90.0/24",
                    "Gateway": "192.168.90.1",
                    "AuxiliaryAddresses": {
                        "dmz-docker-host": "192.168.90.11"
                    }
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "b0a3a365d437130a56e2e9a315255201f407f0489757a2576076da36a36fdec7": {
                "Name": "portainer",
                "EndpointID": "fa6416ef24ef17e131718bb62bf6874f47b99fe3eefe9a9fc9fe4e8a1672054d",
                "MacAddress": "02:42:c0:a8:5a:14",
                "IPv4Address": "192.168.90.20/24",
                "IPv6Address": ""
            }
        },
        "Options": {
            "parent": "dmz"
        },
        "Labels": {}
    }
]

configured as:

$ ip -d link show dev dmz
5: dmz@enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether d0:50:99:a3:b0:84 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 0 maxmtu 65535 
    vlan protocol 802.1Q id 90 <REORDER_HDR> addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 64000 gso_max_segs 64 

$ ip route list table 90
default via 192.168.90.1 dev dmz proto static

So, what went wrong here?

2

OK; I did some more tests:

    docker_host $ hostname -I 
    192.168.1.11 192.168.10.11 192.168.90.11 172.17.0.1 

    docker_host $ docker run -d --rm --name=portainer \
    -v data:/data -v /var/run/docker.sock:/var/run/docker.sock \
    --ip=192.168.90.11 -p 192.168.90.11:80:9000 \
    portainer/portainer

    My_Host $ curl -I 192.168.90.11:80
    HTTP/1.1 200 OK
    Accept-Ranges: bytes
    Cache-Control: max-age=31536000
    Content-Length: 23032
    Content-Type: text/html; charset=utf-8
    Last-Modified: Thu, 19 Mar 2020 22:46:17 GMT
    X-Content-Type-Options: nosniff
    X-Xss-Protection: 1; mode=block
    Date: Wed, 20 May 2020 15:40:32 GMT

So, it binds correct to systemd network’s applied IPs (192.168.10.11 192.168.90.11) but not the docker network/IP ones (192.168.90.20). Hence, the question rise: is there a bug??? Or miss I somethings?

Anyway, netstat shows what I’m expecting.

    $ sudo netstat -tulpen
    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode      PID/Program name   
    tcp        0      0 192.168.90.11:80        0.0.0.0:*               LISTEN      0          26428      1206/docker-proxy   
    tcp        0      0 127.0.0.1:45393         0.0.0.0:*               LISTEN      0          21855      398/containerd      
    tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      98         35976      2502/systemd-resolv 
    tcp        0      0 :::22                   :::*                    LISTEN      0          19558      1/systemd           
    udp        0      0 127.0.0.53:53           0.0.0.0:*                           98         35975      2502/systemd-resolv

This is what inspect got:

    $ docker inspect portainer
    [
        {
                "NetworkMode": "default",
                "PortBindings": {
                    "9000/tcp": [
                        {
                            "HostIp": "192.168.90.11",
                            "HostPort": "80"
                        }
                    ]
                },

            "NetworkSettings": {
                "Bridge": "",
                "SandboxID": "b5b930000ea93b382badf7e8211fbc661dba0ed978009ea0d33f25c82ac20137",
                "HairpinMode": false,
                "LinkLocalIPv6Address": "",
                "LinkLocalIPv6PrefixLen": 0,
                "Ports": {
                    "9000/tcp": [
                        {
                            "HostIp": "192.168.90.11",
                            "HostPort": "80"
                        }
                    ]
                },
                "SandboxKey": "/var/run/docker/netns/b5b930000ea9",
                "SecondaryIPAddresses": null,
                "SecondaryIPv6Addresses": null,
                "EndpointID": "495b3be3e32846f758aea3d1580c287d03f39f0c1e8b8e22de606c43dd059254",
                "Gateway": "172.17.0.1",
                "GlobalIPv6Address": "",
                "GlobalIPv6PrefixLen": 0,
                "IPAddress": "172.17.0.2",
                "IPPrefixLen": 16,
                "IPv6Gateway": "",
                "MacAddress": "02:42:ac:11:00:02",
                "Networks": {
                    "bridge": {
                        "IPAMConfig": null,
                        "Links": null,
                        "Aliases": null,
                        "NetworkID": "3e09ddd10d8952a15f14c1b21e65c0a4e7028919dc8a34607c4f303f2bcb7043",
                        "EndpointID": "495b3be3e32846f758aea3d1580c287d03f39f0c1e8b8e22de606c43dd059254",
                        "Gateway": "172.17.0.1",
                        "IPAddress": "172.17.0.2",
                        "IPPrefixLen": 16,
                        "IPv6Gateway": "",
                        "GlobalIPv6Address": "",
                        "GlobalIPv6PrefixLen": 0,
                        "MacAddress": "02:42:ac:11:00:02",
                        "DriverOpts": null
                    }
                }
            }
        }
    ]

I’m using clear linux with

docker info
Client:
 Debug Mode: false
Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 83
 Server Version: 19.03.8
 Storage Driver: overlay2
  Backing Filesystem: <unknown>
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: systemd
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 
 runc version: 
 init version: 
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.6.13-952.native
 Operating System: Clear Linux OS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.12GiB
 Name: clr1
 ID: HNJ5:IFAK:XMTT:2IIM:LVFV:63ZR:QNFO:SFDS:BF4T:7EUQ:O4TC:PV7S
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
3

I am not sure, if I did see information regarding your local lan ip-range and the type of docker network you use in your post…

Just to be sure: are your aware that you can’t assign ip’s outside the ip range of the docker network you attach the container to?

4

thanks for you help!

I created a picture to show my network settings regards docker. localhost is out of the scope on the picture.
Unfornately, I’m to new to be allowed to upload here … Anyway, here is what I understood/intended:

Bild dockerservernetworkcjkth.png auf abload.de

On DMZ network there is no DHCP (as far I’ve read doesn’t work with macvlan yet). Hence, each service shall get an own static IP on DMZ. I’m not sure about my requirements of different MAC addresses as macvlan provides. IMO ipvlan L3 should work, but this may be the future.

The admin VLAN shall serve SSH once a day, not as today LAN 192.168.1.11 …

Yes, I’m not aware that I try to do this. Without given fixed IP docker DHCP can set IP full range in 192.168.90.0/24 expect docker’s host (and probably the gateway self):

    $ docker network create -d macvlan \
        --subnet=192.168.90.0/24 --gateway=192.168.90.1 \
        --aux-address="dmz-docker-host=192.168.90.11" \
        -o parent=dmz \
        dmz_net

More details on my problems may be found on Unable to attach to macvlan net.

Previous, my network wasn’t separated into DMZ etc. (I mean, I’m still on working this to separate). On docker host I did run jwilder’s reverse nginx, all on 192.168.1.0/24 wihout public internet services. Netxcluod got the first problems (with upload even with smaller ~1MB files) with the proxy (without worked as expected). Now I’m working on getting mail/nextcloud reachable to family even outside the LAN… So things go more professional :slight_smile:

5

Since you already seem to follow instructions from https://github.com/docker/cli/blob/master/experimental/vlan-networks.md.

On you manual attempt, your created and enabled a vlan subinterface, then create the docker network and see that the containers are actualy using the subinterface.

You already create a docker macvlan/ipvlan network, hence are not out of the docker networks range on your experiements. You need to make sure the vlan subinterfaces are created and started after the system starts, before docker is started. Anothing thing to note is, that due to kernel restrictions a macvlan ip is not allowed to communicate with its parent interface - not sure if the same restriction is true for ipvlan as well though.