Docker overlay network without docker-machine

marcosmachado · April 15, 2016, 12:30pm

Hi everyone,

I’m trying to create a swarm cluster with overlay network but, i don’t wanna use docker-machine because i’m using my own hardware and i do not want to use cloud service.

my scenario is:
docker-host-01: host with centos - ip 172.19.232.69
docker-host-02: host with centos - ip 172.19.232.81
docker-host-03: host with centos - ip 172.19.232.91

my docker env(all hosts)

docker version
Client:
 Version:      1.11.0
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   4dc5990
 Built:        Wed Apr 13 18:40:36 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.11.0
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   4dc5990
 Built:        Wed Apr 13 18:40:36 2016
 OS/Arch:      linux/amd64
[root@host-docker-01 marcos]# 


docker info
 WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
Images: 16
Server Version: 1.11.0
Storage Driver: devicemapper
 Pool Name: docker-253:0-50823327-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 1.54 GB
 Data Space Total: 107.4 GB
 Data Space Available: 6.705 GB
 Metadata Space Used: 4.063 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.143 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.107-RHEL7 (2015-12-01)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge null host
Kernel Version: 3.10.0-327.13.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 3
Total Memory: 488.8 MiB
Name: host-docker-01
ID: BBZ3:UZV4:BBCJ:BEZL:KDB2:37A5:7DVP:RYDA:Z6RX:JENV:QZNM:VO6O
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false

uname -a
Linux host-docker-01 3.10.0-327.13.1.el7.x86_64 #1 SMP Thu Mar 31 16:04:38 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

docker -H :4000 info
Containers: 5
 Running: 5
 Paused: 0
 Stopped: 0
Images: 5
Server Version: swarm/1.2.0
Role: primary
Strategy: spread
Filters: health, port, dependency, affinity, constraint
Nodes: 2
docker-host-01: 172.19.232.69:2375
  └ Status: Healthy
  └ Containers: 3
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 513.5 MiB
  └ Labels: executiondriver=, kernelversion=3.10.0-327.10.1.el7.x86_64, operatingsystem=CentOS Linux 7 (Core), storagedriver=devicemapper
  └ Error: (none)
  └ UpdatedAt: 2016-04-15T12:08:52Z
  └ ServerVersion: 1.11.0
docker-host-02: 172.19.232.81:2375
  └ Status: Healthy
  └ Containers: 2
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 513.5 MiB
  └ Labels: executiondriver=, kernelversion=3.10.0-327.10.1.el7.x86_64, operatingsystem=CentOS Linux 7 (Core), storagedriver=devicemapper
  └ Error: (none)
  └ UpdatedAt: 2016-04-15T12:09:03Z
  └ ServerVersion: 1.11.0
Plugins: 
 Volume: 
 Network: 
Kernel Version: 3.10.0-327.10.1.el7.x86_64
Operating System: linux
Architecture: amd64
CPUs: 2
Total Memory: 1.003 GiB
Name: 69eb9543f908
Docker Root Dir: 
Debug mode (client): false
Debug mode (server): false
WARNING: No kernel memory limit support

On docker-host-03

I ran docker

 docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock

I started consul container

 docker run -d -p 8500:8500 --name consul progrium/consul -server -bootstrap

On docker-host-01

I started docker

 docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --cluster-store=consul://172.19.232.91:8500  --cluster-advertise=enp0s3:2375 &

docker swarm manager

 docker run -d --restart=unless-stopped -p 4000:4000 swarm manage -H :4000 --replication --advertise 172.19.232.69:4000  consul://172.19.232.91:8500

add node to cluster

 docker run -d swarm join --advertise=172.19.232.69:2375 consul://172.19.232.91:8500

docker ps
[root@localhost marcos]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS                                                                            NAMES
2dc0a3d99b5f        swarm               "/swarm join --advert"   52 seconds ago       Up 52 seconds       2375/tcp                                                                         condescending_visvesvaraya
edb7ea38875c        swarm               "/swarm manage -H :40"   About a minute ago   Up About a minute   2375/tcp, 0.0.0.0:4000->4000/tcp                                                 drunk_brown
8ccb8fd5ac1e        progrium/consul     "/bin/start -server -"   2 minutes ago        Up 2 minutes        53/tcp, 53/udp, 8300-8302/tcp, 8400/tcp, 8301-8302/udp, 0.0.0.0:8500->8500/tcp   consul

On docker-host-02:

I started docker

 docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --cluster-store=consul://172.19.232.91:8500  --cluster-advertise=enp0s3:2375 &

add node to cluster

 docker run -d swarm join --advertise=172.19.232.81:2375 consul://172.19.232.91:8500

I’ve created a overlay network:

docker -H :4000 network create -d overlay info
docker -H :4000 network create -d overlay info2
docker -H :4000 network create -d overlay teste

on docker-host-01

docker network ls
NETWORK ID          NAME                DRIVER
c22a48c6e1c6        bridge              bridge              
b81f5dd1de16        docker_gwbridge     bridge              
bb2235c6fcf2        host                host                
c8819e13d845        info                overlay             
fd3f780ca95f        info2               overlay             
e16fac408255        none                null                
eac2986e5683        teste               overlay

on docker-host-02

docker network ls
NETWORK ID          NAME                DRIVER
31f5c2bc3d6a        bridge              bridge              
18cc30825c00        docker_gwbridge     bridge              
a8f035e0def0        host                host                
c8819e13d845        info                overlay             
fd3f780ca95f        info2               overlay             
e8215e819164        none                null                
eac2986e5683        teste               overlay

I’ve created web01 and web02 containers on info2 network

docker -H :4000 run -ti -d --name web01 --net info2 nginx
docker -H :4000 run -ti -d --name web02 --net info2 nginx

docker -H :4000 ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
edeeb902ee41        nginx               "nginx -g 'daemon off"   22 seconds ago      Up 21 seconds       80/tcp, 443/tcp     docker-host-02/web02
0313b6327111        nginx               "nginx -g 'daemon off"   15 hours ago        Up 15 hours         80/tcp, 443/tcp     docker-host-01/web01

docker inspect web01
"NetworkSettings": {
            "Bridge": "",
            "SandboxID": "1e235ca08b87fc2ac236077c3608b2210667bf1ada32b1c608da8376ae2c59bf",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "443/tcp": null,
                "80/tcp": null
            },
            "SandboxKey": "/var/run/docker/netns/1e235ca08b87",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "info2": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "fd3f780ca95ff75d016a199209f7b7f18796d1db95aa87a316bffab4957d55d7",
                    "EndpointID": "f154fd78243e1e17c81270660b743f29376fce702571281dcc08e1f196d9b1fc",
                    "Gateway": "",
                    "IPAddress": "10.0.1.3",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:0a:00:01:03"
                }
            }
        }

docker inspect web02
"NetworkSettings": {
            "Bridge": "",
            "SandboxID": "d5aa486e4e8f0f439dff677fccd266acd7d5f87c421a374f214c69ae17acad26",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "443/tcp": null,
                "80/tcp": null
            },
            "SandboxKey": "/var/run/docker/netns/d5aa486e4e8f",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "info2": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "fd3f780ca95ff75d016a199209f7b7f18796d1db95aa87a316bffab4957d55d7",
                    "EndpointID": "2fbd1fa70cbd9ed551786b06dceca729339ba6a792e45a6b9d146e74dd03f843",
                    "Gateway": "",
                    "IPAddress": "10.0.1.4",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:0a:00:01:04"
                }
            }
        }

but when i try docker exec web02 ping web01

docker exec  web02 ping 10.0.1.3
docker exec  web02 ping 10.0.1.4

these comands didn’t work

I forgot some step ?

Topic		Replies	Views
Overlay Network : Unable to ping container from one host to another Docker Hub	0	2559	December 28, 2017
Overlay Network Issues General	3	3231	May 6, 2016
Overlay network problem with openstack / vmware setup Swarm	1	3026	May 24, 2016
Enable to setup an overlay network Swarm	0	879	March 22, 2019
Docker: Unable to ping between nodes using overlay in windows container Docker Desktop docker , swarm	0	1970	March 25, 2019

Docker overlay network without docker-machine

Related topics