Docker Community Forums

Share and learn in the Docker community.

Intermitant networking issues

Docker version 19.03.3, build a872fc2f86
Distributor ID: Debian
Description: Debian GNU/Linux 10 (buster)
Release: 10
Codename: buster

steps to reproduce: This seems to have begun happening after a reboot. No change in software versions happened right before this started happening about a week ago.

The networking stack for my docker containers seems broken. The following snippet is from my bind server trying to start up:

22-Aug-2020 16:17:57.451 socket.c:1731: unexpected error:
22-Aug-2020 16:17:57.451 internal_send: 192.5.5.241#53: Invalid argument
22-Aug-2020 16:17:57.451 socket.c:1731: unexpected error:
22-Aug-2020 16:17:57.451 internal_send: 198.97.190.53#53: Invalid argument
22-Aug-2020 16:17:57.451 socket.c:1731: unexpected error:
22-Aug-2020 16:17:57.451 internal_send: 199.7.91.13#53: Invalid argument
22-Aug-2020 16:17:58.231 socket.c:1731: unexpected error:
22-Aug-2020 16:17:58.231 internal_send: 202.12.27.33#53: Invalid argument

trying just ping from that same container has a similar result:

root@proxmox:/zfspool/DockerVolumes/composefiles/bind9# docker exec -i bind9_bind_1 ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
ping: sendto: Invalid argument

Note that for both of the above, SOMETIMES it will work, about 1 in 10 times. However shortly after it will start failing again:

root@proxmox:/zfspool/DockerVolumes/composefiles/bind9# docker exec -i bind9_bind_1 ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=118 time=1027.318 ms
64 bytes from 8.8.8.8: seq=1 ttl=118 time=27.195 ms
64 bytes from 8.8.8.8: seq=2 ttl=118 time=13.848 ms
64 bytes from 8.8.8.8: seq=3 ttl=118 time=13.640 ms
64 bytes from 8.8.8.8: seq=4 ttl=118 time=13.818 ms
64 bytes from 8.8.8.8: seq=5 ttl=118 time=13.738 ms
64 bytes from 8.8.8.8: seq=6 ttl=118 time=13.667 ms
ping: sendto: Invalid argument

This is happening in all my containers, regardless of type (alpine, ubuntu, etc.)

root@proxmox:/zfspool/DockerVolumes/composefiles/bind9# docker exec -i an_ubuntu_based_container ping 8.8.8.8
ping: sendmsg: Invalid argument
ping: sendmsg: Invalid argument
ping: sendmsg: Invalid argument
ping: sendmsg: Invalid argument

The host machine has no problems with networking, nor do VMs on it. Host machine is Debian 10 running proxmox for VMs.

If I take the exact compose and dockerfiles I used for these containers and move them to another host, they work fine.

I’ve tried restarting the docker service and reinstalling it with apt.

Where can I look next?

I found the cause of this.

My host was complaining about a full arp cache. I raised the limits and my containers appear to be operating fine now.