[Docker] Why the source port of vxlan encapsulated UDP packet is changed occasionally?

General
1

Hi, guys.

I have a problem using vxlan.

Sometimes, the source port of the encapsulated UDP packet is changed.

My data flow likes below

Docker Container <-> Docker Host <-> Load Balancer  <-> NAT <-> Endpoint(HTTP Server)

This source port change issue occurs when there is a latency in receiving a response from the endpoint and also, in processing TCP handshake.

For example when TCP handshaking

[SYN] container port: 33922, mapped host port(UDP source port): 49550
[SYN+ACK] from load balancer (component source port: 40034)
[ACK] container port: 33922, mapped host port(UDP source port): 48173
    - I don't know why the mapped host port is changed and I want to find the solution.
[RST] from load balancer (component source port: 49844)
    - Because of wrong load balance, other components receive [ACK] packet.
[TCP Retransmission][SYN+ACK] from load balancer (component source port: 40034) ← didnot receive [ACK] packet.

Can I know the reason why src port is changed and how to prevent it?

Thank you.

3

Fortunately, I found the reason why the port number of an encapsulated UDP packet was changed. This is because the Linux kernel, which is above 4.x, sometimes changes the port number of the UDP packet to replace the route when TCP retransmission occurs. In my system, there were latencies between the docker container and the endpoint occasionally and it causes the change of the outer port number.

This is the patch about why outer port is changed

This patch creates sk_set_txhash and eliminates protocol specific
inet_set_txhash and ip6_set_txhash. sk_set_txhash simply sets a
random number instead of performing flow dissection. sk_set_txash
is also allowed to be called multiple times for the same socket,
we'll need this when redoing the hash for negative routing advice.