How to debug failures in attaching to overlay network?

General
1

hi

I have created a overlay network and trying to create containers on this network.
But creating the container fails with below error.
Is there anyway to enable debugs and find the exact reason of the failure ?

docker run -d --network overlay --name node-exporter-s101-swarm node-exporter:23.2.0-release 8c10985817b67c35823f3e1ac515bf72f58a1298ba3b2e631fcd06fb176cb750

docker: Error response from daemon: attaching to network failed, make sure your network options are correct and check manager logs: context deadline exceeded.

 ``` docker inspect overlay [ { "Name": "overlay", "Id": "zbvx833xc72283rw53p7syzyf", "Created": "2024-06-04T04:26:18.404475792Z", "Scope": "swarm", "Driver": "overlay", "EnableIPv6": true, "IPAM": { "Driver": "default", "Options": null, "Config": [ { "Subnet": "fd00:3984:3989::/64", "Gateway": "fd00:3984:3989::1" }, { "Subnet": "172.20.0.0/20", "Gateway": "172.20.0.1" } ] }, "Internal": false, "Attachable": true, "Ingress": false, "ConfigFrom": { "Network": "" }, "ConfigOnly": false, "Containers": null, "Options": { "com.docker.network.driver.overlay.vxlanid_list": "4097,4098" }, "Labels": null } ] ```
2

I’m not sure it helps, but dockerd and the docker commands both have the -D or --debug flag and you can check system logs.

sudo journalctl -e -u docker

Since your error message says you should check the manager logs and network options, you should start with that. Maybe there is something wrong with the vxlanid list. How did you create the overlay network?

3

I created the docker_gwbridge manually before creating the swarm using below command.

docker network create
–ipv6
–subnet 172.20.0.0/20
–gateway 172.20.0.1
–gateway fd00:3984:3989::1
–subnet fd00:3984:3989::/64
–opt com.docker.network.bridge.name=docker_gwbridge
–opt com.docker.network.bridge.enable_icc=true
–opt com.docker.network.bridge.enable_ip_forwarding=true
–opt com.docker.network.bridge.enable_ip_masquerade=true
docker_gwbridge

I initialised the swarm using the below command

docker swarm init --advertise-addr 192.168.11.70

I created the overlay network using below command

docker network create --driver=overlay --attachable --ipv6 --gateway fd00:3984:3989::1 --subnet fd00:3984:3989::/64 --subnet 172.20.0.0/20 --gateway 172.20.0.1 overlay

4

I am seeing this error in my logs

Jun 05 03:31:18 an-master dockerd[1102]: time=“2024-06-05T03:31:18.107398595Z” level=info msg="initialized VXLAN UDP port to 4789 "
Jun 05 03:31:18 an-master dockerd[1102]: time=“2024-06-05T03:31:18.323174426Z” level=error msg=“fatal task error” error=“Pool overlaps with other one on this address space” module=node/agent/taskmanager node.id=nktdgxxowvjwrzcbuglvzdrl9 service.id= task.id=bydfo78axydpl8031n8yfjfmz
Jun 05 03:31:18 an-master dockerd[1102]: time=“2024-06-05T03:31:18.393248858Z” level=info msg="initialized VXLAN UDP port to 4789 "

5

with docker daemon debugs enabled, I get these logs

un 05 03:38:13 an-master dockerd[1065]: time=“2024-06-05T03:38:13.685242673Z” level=debug msg=“FIXME: Got an API for which error does not match any expected type!!!: attaching to network failed, make sure your network options are correct and check manager logs: context deadline exceeded” error_type=“*errors.errorString” module=api
Jun 05 03:38:13 an-master dockerd[1065]: time=“2024-06-05T03:38:13.685281004Z” level=error msg=“Handler for POST /v1.41/networks/overlay/connect returned error: attaching to network failed, make sure your network options are correct and check manager logs: context deadline exceeded”
Jun 05 03:38:13 an-master dockerd[1065]: time=“2024-06-05T03:38:13.685299504Z” level=debug msg=“FIXME: Got an API for which error does not match any expected type!!!: attaching to network failed, make sure your network options are correct and check manager logs: context deadline exceeded” error_type=“*errors.errorString” module=api

thanks

6

I edited your first post for a reason. I see you changed it back.

Please, format your posts according to the following guide: How to format your forum posts
In short: please, use </> button to share codes, terminal outputs, error messages or anything that can contain special characters which would be interpreted by the MarkDown filter. Use the preview feature to make sure your text is formatted as you would expect it and check your post after you have sent it so you can still fix it.

Example code block:

```
echo "I am a code."
echo "An athletic one, and I wanna run."
```