Docker containers with "public" IPs, bridged network

General
1

I basically followed this guide: Docker Containers with Public IPs

We already have a similar setup working in another location, but I can’t get it working in a new environment. Sadly, my predecessor hasn’t documented anything, so im trying to reverse engineer the setup.

Docker Host: 10.10.60.41/24

with docker bridged network: docker network create --subnet=10.60.0.0/16 --opt "com.docker.network.bridge.name"="br-ext" ext

routes on docker host:

#  ip r
default via 10.10.60.1 dev br0 proto static 
10.10.60.0/24 dev br0 proto kernel scope link src 10.10.60.41 
10.60.0.0/16 dev br-ext proto kernel scope link src 10.60.0.1 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdow

run a docker container: docker run --network=ext -itd --name=web nginx

That docker container gets IP 10.60.0.2 assigned.

ping 10.60.0.2 or curl 10.80.0.2 from the docker host is working fine…as expected.

But the docker container is not reachable from the network. A network route for 10.60.0.0/16 to the primary IP of the docker host 10.10.60.41 is set.

# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
DOCKER-USER  all  --  anywhere             anywhere            
DOCKER-ISOLATION-STAGE-1  all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain DOCKER (2 references)
target     prot opt source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target     prot opt source               destination         
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere            
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
target     prot opt source               destination         
DROP       all  --  anywhere             anywhere            
DROP       all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            

Chain DOCKER-USER (1 references)
target     prot opt source               destination         
RETURN     all  --  anywhere             anywhere

# iptables -t nat -L -n -v
Chain PREROUTING (policy ACCEPT 35363 packets, 2140K bytes)
 pkts bytes target     prot opt in     out     source               destination         
 140K 8413K DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 24828 packets, 1495K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 286 packets, 19813 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    6   504 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 10799 packets, 659K bytes)
 pkts bytes target     prot opt in     out     source               destination         
    6   504 MASQUERADE  all  --  *      !br-ext  10.60.0.0/16         0.0.0.0/0           
    0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0           

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    2   168 RETURN     all  --  br-ext *       0.0.0.0/0            0.0.0.0/0           
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0

# sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

The two setups are basically identical, except subnets etc. But it looks like I’m missing something here…any help would be greatly appreciated.

Thanks in advance and have a nice day!

2

just a liitle follow up,

indeed this works. Only thing is to look after iptables rules…
had to add something like this:

iptables -A FORWARD -s 10.10.60.0/24 -d 10.60.0.0/16 -j ACCEPT

this, also should work:

iptables -P FORWARD ACCEPT

additional info…
on Cloud platforms like Openstack, there is another problem i stumpled upon:
Subnets defined locally on the instances are unknown to Openstack and are not routed between the VMs. i didn’t know that…
you have to add “Allowed Address Pairs”

"openstack port set PORT_UUID --allowed-address \
ip-address=<IP_CIDR>[,mac_address=<MAC_ADDRESS]"