Packet drop over overlayNetwork in docker swarm mode

Scenario as follows

To achieve Load-balanceing udp, syslog ,across two service

Network Device ip: 192.168.10.1
Two nodes

  • node-1 ip : 192.168.20.245
  • node-2 ip : 192.168.20.246

Centos with kernel 4.14

Swarm mode

Have two service with python application running in it on port 514 (syslog) , no port forwarding and deployed using docker stack deploy across two node with “mode: global” and "endpoint_mode: dnsrr " set and default overlay network created with “attachable: true” .

Service containers ip
syslog-01.xxxx : 10.0.9.2
syslog-02.xxxx : 10.0.9.3

Have another service with ipvs running to load-balance syslog across above two services over overlay-Network , deployed using docker-compose -f compose.yml up -d .
And this attaches to above overlay network with port forwarding on 514 .

ipvs load-balancing as expected .
Note: Source ip of the udp will not change ,

Issue :
I can see packet received on container NIC level using tcpdump but application can’t see it .
Output of tcpdump on container syslog-01.xxxx

 20:01:29.403268 IP 172.16.111.1.514 > 10.0.9.2.514: SYSLOG user.info, length: 60
 
 Output of tcpdump on container syslog-01.xxxx

 22:09:30.686667 IP 172.16.111.1.514 > 10.0.9.3.514: SYSLOG user.info, length: 66

 Observation : No iptable filter table entry seen in container network name space and overlay name space and If  generate syslog on ipvs container using python it get received on syslog-01 and syslog-02 containers 

I presume packet getting dropped somewhere in transit not sure

Please Help
Please let me know if more information required
Thank you in advance