Docker and iptables configuration @startup

For a while now I’ve been facing problems connecting to the outside world from within my images. I think I have pin-pointed the root cause of this. First let me describe the situation:

  • Ubuntu server, running 14.04.2 LTS
  • Two physical NICs: eth0 (internet) and p1p1 (local)
  • The server is acting as a router for my internal network. The configuration of the iptables can be found below. Of course /proc/sys/net/ipv4/ip_forward is set to TRUE.
  • in /etc/defaults/docker I have set the following options: DOCKER_OPTS="--ip-masq=true --dns 8.8.8.8 --dns 8.8.4.4"

Reproducing the problem
Now when I start an image, f.e. this one: docker run -i -t ubuntu:14.04 /bin/bash and I’m trying to ping google.com, I receive a destination unknown. Some Googling taught me that it has to do with masqueradring not being enabled. So I checked the rules by running this: sudo iptables -t nat -L -n. And indeed, the output is as follows:

> ~/ -> sudo iptables -t nat -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  0.0.0.0/0            0.0.0.0/0

Basically this means that there is no NAT possible between a Docker container and “the internet”. Now when I restart Docker (sudo service docker restart), the missing rules are added:

> ~/ -> sudo iptables -t nat -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
DOCKER     all  --  0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
DOCKER     all  --  0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  172.17.0.0/16        0.0.0.0/0
MASQUERADE  all  --  0.0.0.0/0            0.0.0.0/0

Chain DOCKER (2 references)
target     prot opt source               destination

My conclusion
My conclusion is that there is some kind of problem in the order at bootup. I suppose the docker daemon get’s started first (and adds the missing rules) and THEN the iptables are being restored. This is done through the /etc/rc.local file: iptables-restore < /etc/iptables.sav. I also find this post, which basically describes my problem, but in fewer words :slight_smile: http://stackoverflow.com/questions/25917941/docker-how-to-re-create-dockers-additional-iptables-rules

IPTABLES configuration script

#!/bin/sh
# p1p1 = internal NIC
# eth0 = WAN NIC
lan_nic=p1p1
wan_nic=eth0

echo Configuration: [lan_nic = $lan_nic], [wan_nic = $wan_nic]

echo "Clear old firewall rules..."
iptables --flush
iptables --flush FORWARD
iptables --flush INPUT
iptables --flush OUTPUT
iptables --table nat --flush
iptables --table nat --delete-chain
iptables --table mangle --flush
iptables --table mangle --delete-chain
iptables --delete-chain

echo "Drop all INPUT and FORWARD..."
iptables -P INPUT DROP
iptables -P FORWARD DROP

echo "Drop all IPv6 traffic..."
ip6tables -P INPUT DROP
ip6tables -P OUTPUT DROP
ip6tables -P FORWARD DROP

echo "Accept everything on lo and $lan_nic (local network)..."
iptables -A INPUT -i lo -p all -j ACCEPT
iptables -A OUTPUT -o lo -p all -j ACCEPT
iptables -A INPUT -i $lan_nic -p all -j ACCEPT -s 192.168.1.0/24
iptables -A OUTPUT -o $lan_nic -p all -j ACCEPT -d 192.168.1.0/24

echo "IP Forwarding and Routing for gateway use..."
iptables -A FORWARD -o $wan_nic -i $lan_nic -s 192.168.1.0/24 -m conntrack --ctstate NEW -j ACCEPT
iptables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A POSTROUTING -o $wan_nic -j MASQUERADE

echo "Maintain established connections..."
iptables -A INPUT --in-interface $wan_nic --match conntrack --ctstate ESTABLISHED,RELATED --jump ACCEPT

echo "Allow port 80 (http) connections to the firewall..."
iptables -A INPUT -i $wan_nic -p tcp --dport 80 -m state --state NEW -j ACCEPT
2 Likes

Thanks a lot, you saved my day.

So these are the rules that I’m missing on iptables after docker restarts:

:PREROUTING ACCEPT [8:496]		
:INPUT ACCEPT [0:0]		
:OUTPUT ACCEPT [0:0]		
:POSTROUTING ACCEPT [0:0]		
:DOCKER - [0:0]		
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER		
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER		
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE		
-A POSTROUTING -s 172.18.0.0/16 ! -o br-a0b355ce53ac -j MASQUERADE		
-A DOCKER -i docker0 -j RETURN		
-A DOCKER -i br-a0b355ce53ac -j RETURN
 # same
:DOCKER - [0:0]		
:DOCKER-ISOLATION - [0:0]
# same
	-A FORWARD -j DOCKER-ISOLATION		
-A FORWARD -o docker0 -j DOCKER		
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT		
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT		
-A FORWARD -i docker0 -o docker0 -j ACCEPT		
-A FORWARD -o br-a0b355ce53ac -j DOCKER		
-A FORWARD -o br-a0b355ce53ac -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT		
-A FORWARD -i br-a0b355ce53ac ! -o br-a0b355ce53ac -j ACCEPT		
-A FORWARD -i br-a0b355ce53ac -o br-a0b355ce53ac -j ACCEPT
# same
	-A DOCKER-ISOLATION -i br-a0b355ce53ac -o docker0 -j DROP		
-A DOCKER-ISOLATION -i docker0 -o br-a0b355ce53ac -j DROP		
-A DOCKER-ISOLATION -j RETURN

I see the problem but I don’t see solution - how can we control the boot order? BTW -restarting docker doesn’t help because my iptables=false