Hello everyone,
I have trouble tranferring a solution from AWS EC2 instances to OVH.
This solution uses Docker Swarm. Initially, some problems occured because the VXLAN option was not set on the server, but I got that fixed.
Nevertheless, my containers can’t seem to be able to communicate with one another. After some investigation, I found that one or several container have multiple IP address but I can’t determine why.
My setup is :
$ uname -a
Linux <hostname> 4.9.135-xxxx-std-ipv6-64 #1 SMP Mon Oct 22 13:00:30 UTC 2018 x86_64 GNU/Linux
$ docker version
Client:
Version: 18.06.1-ce
API version: 1.38
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:23:18 2018
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 18.06.1-ce
API version: 1.38 (minimum version 1.12)
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:22:21 2018
OS/Arch: linux/amd64
Experimental: false
$ docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 1
Server Version: 18.06.1-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: active
NodeID: <SNIP>
Is Manager: true
ClusterID: <SNIP>
Managers: 3
Nodes: 3
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 10
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: <SNIP>
Manager Addresses:
<SNIP>:2377
<SNIP>:2377
<SNIP>:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 468a545b9edcd5932818eb9de8e72413e616e86e
runc version: 69663f0bd4b60df09991c08812a60108003fa340
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.135-xxxx-std-ipv6-64
Operating System: Debian GNU/Linux 9 (stretch)
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 31.38GiB
Name: <SNIP>
ID: <SNIP>
Docker Root Dir: /home/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
The stack I am trying to launch is an Hyperledger Fabric’s CA which worked like a charm on AWS EC2 instances. Here’s the output of my intermediate CA :
Post https://ca_root:7054/enroll: dial tcp 10.0.9.7:7054: getsockopt: no route to host
So I inspected inside the container ca_root via several ways :
$ docker inspect ca_ca_root.1.47n1a736rvl1p829b8741j8w3
...
"NetworkSettings": {
"Bridge": "",
"SandboxID": "59c545167c5978679185b5398c44b06b9a18c54cd5df767213f3e8724636d0df",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"7054/tcp": null
},
"SandboxKey": "/var/run/docker/netns/59c545167c59",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"ca_default": {
"IPAMConfig": {
"IPv4Address": "10.0.9.8"
},
"Links": null,
"Aliases": [
"66aa53da2aeb"
],
"NetworkID": "wq39rq1oza2cr4ey95thtwbi4",
"EndpointID": "308fa4122eeb1320f812168597cea5954624ca886a54d24e14d4214d42045725",
"Gateway": "",
"IPAddress": "10.0.9.8",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:0a:00:09:08",
"DriverOpts": null
}
}
}
...
So IP address is 10.0.9.8, not 10.0.9.7 … and it’s been confirmed inside the network :
$ docker network inspect ca_default
[
{
"Name": "ca_default",
"Id": "wq39rq1oza2cr4ey95thtwbi4",
"Created": "2018-11-09T13:48:30.97700976Z",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "10.0.9.0/24",
"Gateway": "10.0.9.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
<SNIP>
"66aa53da2aebacdb176140230f1f4abb40c39f60307600d17fca894487866bfa": {
"Name": "ca_ca_root.1.47n1a736rvl1p829b8741j8w3",
"EndpointID": "308fa4122eeb1320f812168597cea5954624ca886a54d24e14d4214d42045725",
"MacAddress": "02:42:0a:00:09:08",
"IPv4Address": "10.0.9.8/24",
"IPv6Address": ""
}
<SNIP>
"lb-ca_default": {
"Name": "ca_default-endpoint",
"EndpointID": "c931f0c8053966ed242d32824b9b5936bf24c9957ec1557bdeec586b0ef0aad2",
"MacAddress": "02:42:0a:00:09:04",
"IPv4Address": "10.0.9.4/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "4105"
},
"Labels": {
"com.docker.stack.namespace": "ca"
},
"Peers": [
<SNIP>
]
}
]
I tried to get the IP address inside the container :
$ docker exec -it ca_ca_root.1.47n1a736rvl1p829b8741j8w3 /bin/bash
...
# hostname -I
10.0.9.8 172.19.0.4
Once again, I obtained 10.0.9.8 … but then, I tried :
$ docker service inspect ca_ca_root
"Endpoint": {
"Spec": {
"Mode": "vip"
},
"VirtualIPs": [
{
"NetworkID": "wq39rq1oza2cr4ey95thtwbi4",
"Addr": "10.0.9.7/24"
}
]
}
Here is the 10.0.9.7 IP address …
I Honestly do not know how to reproduce this bug and I can not share the docker-compose file.
I already tried to find a solution but they were inapplicable or outdated. Can you help me on this one?
Thanks a lot