Custom Bridge networks lose access to outside internet after server/docker restart

Docker Engine General
, ,
1

Hello everyone, yesterday I noticed that I couldn’t reach my websites anymore so I decided to do some troubleshooting.
Before everything started I was running a two year old install of Fedora 41.
Here are all the things I tried:

  • Restarting the server → no change
  • Upgrading to Fedora 42 and all packages → no change
  • Fresh installing the OS and Docker → no change
  • Resetting and flushing all IPTABLES/firewalld rules → no change
  • Pulling down all of my docker composes deleting the network and recreating it → It actually worked! For a bit at least, after restarting docker/the server everything stopped working again.
  • Restarting my modem → no change

This issue is really strange because the default bridge network Docker creates works without any problems, it’s just the custom ones that just don’t keep working.

I’m pretty sure It’s not related to DNS as when I try to ping google.com from a random Traefik container I can actually see it resolving the IP, it just cannot reach it. The same IP from the host or from the default bridge network pings without any issue.

I looked at iptables before and after restarting docker but the rules do not change so it can’t be that either, otherwise it wouldn’t have worked after recreating the network before rebooting.

I pretty much tried everything I knew. Even nshoot couldn’t give me any insight (I don’t know how to use it properly so maybe there is a tool in there that could help me.

So, am I going crazy? Is my server cursed? Maybe something with se-linux? I don’t know, maybe it’s something with Fedora itself Docker suddenly doesn’t like.

Forgot to add, from the host I can actually reach the containers, if I do localhost:80 i do actually reach Traefik, that always works. Also weird thing, I can ping the host local Ip address (192.168.1.104) But I cannot ping the gateway (192.168.1.1)

I’m sorry if the post is a bit long but I don’t know where to smash my head into anymore.

Thanks in advance for any reply!

2

Ran into this after updating to F42 with Docker version 28.1.1, build 4eba377.
Found this podman issue suggesting a change to the forward policy and it worked!

iptables-save >> ~/iptables_$(date '+%Y-%m-%d_%H-%M')
iptables -P FORWARD ACCEPT

My F41 boxes with the same docker version work just fine, so maybe this is a Fedora problem?
iptables 1.8.10 vs 1.8.11?
:man_shrugging: Above my paygrade

3

While this works, it may be a bit broad/insecure. For me, the culprit was the update to fedora 42 wiped out my changes to /etc/firewalld/firewalld.conf, specifically setting the backend to iptables.

Renable iptables by updating the value for FirewallBackend:

FirewallBackend=iptables

Then restart firewalld:

sudo systemctl restart firewalld

2 Likes
4

Thank you so much — I spent 3 hours debugging why internet stopped working in my containers after upgrading Fedora.

5

Thank you so much for this! I’ve been troubleshooting for the last 2 days trying to get my docker containers working again and just realized that fedora 42 was the cause. Hopefully nftables gets fixed before they fully remove iptables as an option.

6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.