Intermittent routing failure on overlay network in 18.03.1-ce

General
1

Just updated docker this morning and this started to happen:
support@node2:~$ docker container exec -it test-server20 bash
1864565f1aca:/# ping db-server20
PING db-server20 (10.0.1.33) 56(84) bytes of data.
64 bytes from db-server20.legacy_net (10.0.1.33): icmp_seq=1 ttl=64 time=0.934 ms
64 bytes from 323e8fe2a14f.legacy_net (10.0.1.33): icmp_seq=2 ttl=64 time=0.632 ms
64 bytes from db-server20.legacy_net (10.0.1.33): icmp_seq=3 ttl=64 time=0.552 ms
^C
— db-server20 ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 0.552/0.706/0.934/0.164 ms
1864565f1aca:/# ping db-server20
PING db-server20 (10.0.1.32) 56(84) bytes of data.
From 1864565f1aca (10.0.1.36) icmp_seq=1 Destination Host Unreachable
From 1864565f1aca (10.0.1.36) icmp_seq=2 Destination Host Unreachable
From 1864565f1aca (10.0.1.36) icmp_seq=3 Destination Host Unreachable

In my app it looks like a database connection error, just click reload, once or twice, and then it’s back. I have three overlay networks for different purposes. More than one has been showing this problem since the update. Yesterday it was perfectly fine. I plan to attempt a rollback to the previous version. Any other ideas?

2

Reading back my own post I notice the IP addresses for the db container are changing, maybe the overlay network is keeping multiple address for containers that have been stopped, removed, and run again. I’ll try to bring it all down and restart the overlay network.