UDP stream timeout

General
1

Hello,

I’ve come to a problem. When running a Californium COAP server inside a container the UDP stream timeouts after approx. 3 minutes of inactivity. I’ve already extended the timeout on the host side and also on the docker container itself using: sysctl -w net.netfilter.nf_conntrack_udp_timeout_stream=86400.
I then monitored the /proc/net/nf_conntrack and saw that the connection is extended to 86400s.
The connection is also seen in netstat.
But after 3 minutes of inactivity the stream drops. It is seen in /proc/net/nf_conntrack but not anymore in netstat.
Is there a simple setting that I missed or owerlooked?

Thank you in advance!

2

@anej - did you ever find a solution to this. I’m running into similar problems trying to run leshan / californium in a docker container and I’m seeing similar behaviour.

In my case, I have a basic test script which will listen on a datagram socket, and echo back the content of the packet to the sender after a period of time. I see that this delay can be up to 89 seconds. After that (90+ seconds), the response never makes it back to the client.

3

I don’t think that on a Linux/TCP level there is a concept of a “UDP stream”. UDP packets are “connectionless” and “unreliable”, they are just sent from one IP to another IP:port.

As far as I know, there isn’t even a “response” possible like with an incoming TCP connection, but you can only send an independent response packet back to an IP:port.

So the sending and receiving sides need to make sure they continuously list for UDP packets on a port.

4

You’re correct about UDP packets being conceptually connectionless, but docker has a bunch of networking infrastructure between your container and the host which is likely what the issue is. After some more testing, I can say that this doesn’t happen when I take docker out of the picture or when using host mode networking (but since I’m primarily wanting to use this on a mac, host networking isn’t an option). This leads me to believe it’s some sort of timeout within the docker NAT mechanism. For UDP this seems to time out at the 90 second mark with current versions of docker (Docker Desktop 4.30.0 as of writing).

This can be exercised with the following basic python code (I’m in a python3.9 container, but the original issue is with a java app). The server needs to be in a container, the client does not.

# basic echo server with a delay
import socket
import time

UDP_IP = "::"
UDP_PORT = 5005

sock = socket.socket(socket.AF_INET6,  # Internet
                     socket.SOCK_DGRAM)  # UDP
sock.bind((UDP_IP, UDP_PORT))

while True:
    data, addr = sock.recvfrom(1024) # buffer size is 1024 bytes
    print("received message: %s" % data)
    #time.sleep(900)     # 15 minutes works perfectly on the local machine outside docker. Also works on a raspberry pi running outside docker or in host network mode
    time.sleep(90)    # this does not work in docker (default bridge)
    # time.sleep(89)      # this works in docker (89 works)
    print("sending %s" % data)
    sock.sendto(data, addr)

# Basic Echo Client
import socket

UDP_IP = "::1"
UDP_PORT = 5005
MESSAGE = b"Hello, World!"

print("UDP target IP: %s" % UDP_IP)
print("UDP target port: %s" % UDP_PORT)
print("message: %s" % MESSAGE)

sock = socket.socket(socket.AF_INET6, # Internet
                     socket.SOCK_DGRAM) # UDP
sock.sendto(MESSAGE, (UDP_IP, UDP_PORT))
data, addr = sock.recvfrom(1024) # buffer size is 1024 bytes
print("received message: %s" % data)

So, with all this set, is there a way for me to set any properties which would affect the docker NAT timeouts for UDP streams?

5

You could create an issue on Github.

6

I’ve created UDP Response Timeout in Bridge Mode Networking · Issue #47879 · moby/moby · GitHub

1 Like