Node do not join as Manager just on Google Cloud and Oracle VPC

Hello everyone!! I need some help!!

My scenario have 7 nodes, 4 running in AWS (each one in a different account), 1 running in LINODE, 1 running in Google Cloud and 1 running in Oracle Cloud. Every node is using external IP, and I checked firewall ports into the provider and ensure that is disabled on the VM. I also edited the hosts files in each node to ensure that they will be reacheable, all they are pinging ok.

All machines running in AWS and Linode can join the SWARM both as worker or as manager, but the machines running in the Google Cloud and Oracle, just can join as worker…

Using one AWS node as Leader, I got errors that I show in the image…

One of mine guess was the Linux version, on Google and Oracle I’m using Ubuntu, so I created a new VPN on AWS using the same Ubuntu 18.04 LTS, and in AWS it worked like a charm! For testing I created an account at Linode and up a new VPC also with Ubuntu, and again… worked like a charm!

As my last try, I isolated the Google Cloud and Oracle VPC, into Google Cloud machine I created a new swarm, and tried to join the Oracle machine as manager, and I got a new error, later I took of Linode machine from the original swarm and tried to join into this new one, and again got error…

In this last try, both nodes do not appears on “docker node ls” from leader, and when I try join again the nodes says that they already are into a swarm, so I need use “docker swarm leave” before…

Nodes from Google, Oracle and Linode “docker info”

Linode

Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 19.03.4
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: error
  NodeID: 
  Error: rpc error: code = DeadlineExceeded desc = context deadline exceeded
  Is Manager: false
  Node Address: 45.33.96.86
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 4.15.0-50-generic
 Operating System: Ubuntu 18.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 985.4MiB
 Name: node-7-linode.ip-45-33-x-86
 ID: EJII:OT5T:OSRJ:IKTW:GNAE:XWNB:HHQR:JB5D:PP3I:AQWY:Q42G:H4GI
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Oracle

Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 19.03.4
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: error
  NodeID: 
  Error: rpc error: code = DeadlineExceeded desc = context deadline exceeded
  Is Manager: false
  Node Address: 10.0.0.2
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 4.15.0-1027-oracle
 Operating System: Ubuntu 18.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 982.2MiB
 Name: node-5-oracle.ip-140-238-x-235
 ID: QXNM:APBF:3HGR:7W2Y:IGZQ:2NW7:2UQW:W5CZ:WUFD:YCSM:JA53:JOWN
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Google Cloud

Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 19.03.4
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: active
  NodeID: tjbpsj7xzkcp48ru1ok4dxbvx
  Is Manager: true
  ClusterID: 02wl9fhql0ejx0pzp70iod1t4
  Managers: 1
  Nodes: 1
  Default Address Pool: 10.0.0.0/8  
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 35.212.x.41
  Manager Addresses:
   35.212.x.41:2377
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.0.0-1021-gcp
 Operating System: Ubuntu 18.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 579.1MiB
 Name: node-6-gcloud.ip-35-212-x-41
 ID: VMF7:ERMW:4O6O:HFIY:T24W:QBN5:EQEO:IEYW:Z7VE:DK4D:G3OQ:YPLD
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false