Memberlist: handler queue full, dropping message (8)

Hi docker experts,

I am running a docker swarm with 130 worker nodes and 1 manager.

When deploying a large number of containers (>15.000), I am experiencing the following error messages:

Nov 14 15:22:15 nist_1 dockerd: time=“2022-11-14T15:22:15.718691615+01:00” level=warning msg=“memberlist: handler queue full, dropping message (8) from=192.168.1.55:7946”
Nov 14 15:22:15 nist_1 dockerd: time=“2022-11-14T15:22:15.719623838+01:00” level=warning msg=“memberlist: handler queue full, dropping message (8) from=192.168.0.251:7946”
Nov 14 15:22:15 nist_1 dockerd: time=“2022-11-14T15:22:15.719723190+01:00” level=warning msg=“memberlist: handler queue full, dropping message (8) from=192.168.1.46:7946”

I am also observing the following errors:

Nov 14 15:22:14 nist_1 dockerd: time=“2022-11-14T15:22:14.384838565+01:00” level=warning msg=“memberlist: Failed fallback TCP ping: timeout 1s: read tcp 192.168.0.10:60762->192.168.1.56:7946: i/o timeout”

Where can I find this handler queue and how can I adjust the respective parameter to increase the size of that queue?

Cheers,
Nils