Docker Community Forums

Share and learn in the Docker community.

Connect to a swarm cluster

swarm

(Dahu29) #1

Hello,

I 'm new with docker-swarm.
I’m working on W7 with docker last version of toolbox18.02.0-ce (client), 17.12.1-ce (server)
So for a school project, i’m working with aerospike with docker swarm to improve replication database with different nodes.
I created 3 nodes : `

NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
default - virtualbox Running tcp://192.168.99.100:2376 v17.12.1-ce
nodetest-1 - virtualbox Running tcp://192.168.99.101:2376 v18.05.0-ce
nodetest-2 - virtualbox Running tcp://192.168.99.102:2376 v18.05.0-ce
nodetest-3 * virtualbox Running tcp://192.168.99.103:2376 v18.05.0-ce
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
ddz20vauyqs75qrgzo4ikkcj9 * nodetest-1 Ready Active Leader
lnrni0pc23lblivbpm5cwtg09 nodetest-2 Ready Active
53suwp2tlnkpibpzy3js3yw0z nodetest-3 Ready Active

`I deployed on them this yml file

version: ‘3.2’

services:
aerospikedb:
image: aerospike/aerospike-server:latest
networks:
- aerospikenetwork
deploy:
replicas: 4
endpoint_mode: dnsrr
# Deploiement seulement sur les noeuds workers
placement:
constraints: [node.role == worker]
labels:
com.aerospike.description: “This label is for all containers for the Aerospike service”
command: [ “–config-file”,"/run/secrets/aerospike.conf"]
secrets:
- source: conffile
target: aerospike.conf
mode: 0440
meshworker:
image: aerospike/aerospike-tools:latest
networks:
- aerospikenetwork
depends_on:
- aerospike
entrypoint:
- /run/secrets/discovery
- “–servicename”
- aerospikedb
- “-i”
- “5”
- “-v”
secrets:
- source: discoveryfile
target: discovery
mode: 0750
visualizer:
image: dockersamples/visualizer:stable
ports:
- “8080:8080”
stop_grace_period: 1m30s
volumes:
- “/var/run/docker.sock:/var/run/docker.sock”
deploy:
placement:
constraints: [node.role == manager]
amc:
image: apsops/aerospike-amc:v4.0.19
ports:
- “8081:8081”
deploy:
placement:
constraints: [node.role == manager]
networks:
aerospikenetwork:
driver: overlay
attachable: true

secrets:
conffile:
file: ./aerospike.conf
discoveryfile:
file: ./discovery.py

You will find below the resultat of netstat running on my node leader :

docker@nodetest-1:~$ netstat -tunap
netstat: showing only processes with your user ID
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 0 192.168.99.101:52638 192.168.99.103:7946 TIME_WAIT -
tcp 0 0 10.0.2.15:52834 104.18.122.25:443 ESTABLISHED -
tcp 0 0 192.168.99.101:49658 192.168.99.102:7946 TIME_WAIT -
tcp 0 0 192.168.99.101:49646 192.168.99.102:7946 TIME_WAIT -
tcp 0 0 10.0.2.15:40582 104.18.121.25:443 ESTABLISHED -
tcp 0 0 192.168.99.101:49652 192.168.99.102:7946 TIME_WAIT -
tcp 0 0 10.0.2.15:22 10.0.2.2:63891 ESTABLISHED -
tcp 0 0 10.0.2.15:22 10.0.2.2:56525 ESTABLISHED -
tcp 0 0 10.0.2.15:34370 104.18.123.25:443 ESTABLISHED -
tcp 0 0 192.168.99.101:52632 192.168.99.103:7946 TIME_WAIT -
tcp 0 0 :::2376 :::* LISTEN -
tcp 0 0 :::2377 :::* LISTEN -
tcp 0 0 :::7946 :::* LISTEN -
tcp 0 0 :::8081 :::* LISTEN -
tcp 0 0 :::22 :::* LISTEN -
tcp 0 0 ::ffff:192.168.99.101:7946 ::ffff:192.168.99.102:34592 TIME_WAIT -
tcp 0 0 ::ffff:192.168.99.101:7946 ::ffff:192.168.99.103:56182 TIME_WAIT -
tcp 0 0 ::ffff:192.168.99.101:2377 ::ffff:192.168.99.103:56070 ESTABLISHED -
tcp 0 0 ::ffff:192.168.99.101:7946 ::ffff:192.168.99.102:34602 TIME_WAIT -
tcp 0 0 ::ffff:192.168.99.101:7946 ::ffff:192.168.99.103:56190 TIME_WAIT -
tcp 0 0 ::ffff:192.168.99.101:2377 ::ffff:192.168.99.102:58270 ESTABLISHED -
udp 0 0 0.0.0.0:4789 0.0.0.0:* -
udp 0 0 :::7946 :::* -

Port to access to database is 3000 but the port is not map.

Boot2Docker version 18.05.0-ce, build HEAD : b5d6989 - Thu May 10 16:35:28 UTC 2018
Docker version 18.05.0-ce, build f150324
docker@nodetest-2:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8c6c862d1cac aerospike/aerospike-server:latest “/entrypoint.sh --co.” 10 minutes ago Up 10 minutes 3000-3003/tcp aerospike_aerospikedb.4.t8vy2akn1t93y5kook9knkqsd
166d68edfaa8 aerospike/aerospike-server:latest “/entrypoint.sh --co.” 16 minutes ago Up 16 minutes 3000-3003/tcp aerospike_aerospikedb.2.19xen8a2w2tsxtk9kwwrdhbla

I tried to delete parameter endpoint_mode: dnsrr and add port 3000 but i didn’t success to connect to my data base.
I tried to add HaProxy as a loadbalancer but access is still KO
My purpose is to connect at an ip and port to load data.

How could i modify my yml to success connect to my database ?

Thx


(Etienc) #2

In the compose file you need to add the port mapping. It should look something like this:

ports:
- “3000:3000”

(Dahu29) #3

It’s still KO
I modify my yml file by :

version: ‘3.6’

services:
aerospikedb:
image: aerospike/aerospike-server:latest
networks:
- aerospikenetwork
ports:
- 3000
environment:
- SERVICE_PORTS=3000

And i add these lines for HAPROXY

proxy:
    image: dockercloud/haproxy
    # Won't start until at least one of our app services is up and running.
    depends_on:
      - aerospikedb
    environment:
      # The type of load balancing strategy that will be used.
      # - leastconn sends request to the service with the least active requests.
      # - roundrobin rotates the requests around the services.
      - BALANCE=leastconn
      # Used to identify services.
      #- ADDITIONAL_SERVICES=project_dir:app
      - STATS_PORT=1936
      - STATS_AUTH="admin:admin"
    volumes:
      # Since our app services are running on the same port,
      # the HAProxy will use the docker.sock to find the
      # services that it should load balance.
      - /var/run/docker.sock:/var/run/docker.sock
    ports:
      # The internal used by the HAProxy is 80,
      # but we can expose any port that we would like externally.
      # For example, if you are running something else on 80,
      # you probably don't want to expose the HAProxy on 80 as well.
      - 80:80
      - 1936:1936
      #- 8080:8083
     # - 8081:8084
    networks:
      - aerospikenetwork
    deploy:
      # The HAProxy is assigned as the manager.
      placement:
        constraints: [node.role == manager]

Now i have on docker service ls

$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
elc8dk3eye03 aerospike_aerospikedb replicated 2/2 aerospike/aerospike-server:latest *:30002->3000/tcp
kgegc02bc5gh aerospike_amc replicated 1/1 apsops/aerospike-amc:v4.0.19 *:8081->8081/tcp
saz87m7k1w6j aerospike_meshworker replicated 1/1 aerospike/aerospike-tools:latest
zbynqtjl5o8k aerospike_proxy replicated 1/1 dockercloud/haproxy:latest *:80->80/tcp, *:1936->1936/tcp
26e76c79z4mw aerospike_visualizer replicated 1/1 dockersamples/visualizer:stable *:8080->8080/tcp

I don’t know why i have 30002->3000 instead of 3000

docker@nodetest-1:~$ netstat -tunap
netstat: can’t scan /proc - are you root?
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 304 10.0.2.15:22 10.0.2.2:64145 ESTABLISHED -
tcp 0 0 192.168.99.101:58606 192.168.99.101:2377 ESTABLISHED -
tcp 0 0 :::2376 :::* LISTEN -
tcp 0 0 :::2377 :::* LISTEN -
tcp 0 0 :::7946 :::* LISTEN -
tcp 0 0 :::1936 :::* LISTEN -
tcp 0 0 :::80 :::* LISTEN -
tcp 0 0 :::8080 :::* LISTEN -
tcp 0 0 :::8081 :::* LISTEN -
tcp 0 0 :::30002 :::* LISTEN -
tcp 0 0 :::22 :::* LISTEN -
tcp 0 0 ::ffff:192.168.99.101:2377 ::ffff:192.168.99.101:58606 ESTABLISHED -
udp 0 0 0.0.0.0:4789 0.0.0.0:* -
udp 0 0 :::7946 :::* -

With a new netstat -tunap, port 30002 is expose and not 3000.
But curl test is still KO.

$ curl 192.168.99.101:30002
curl: (7) Failed to connect to 192.168.99.101 port 30002: Connection refused

How can i expose the port with my yml file ?


(Dahu29) #4

I tested to install directly a container aerospike in my node with this command

docker run -tid --name aerospike -p 3000:3000 -p 3001:3001 -p 3002:3002 -p 3003:3003 aerospike/aerospike-server

Access work correctly.

$ curl 192.168.99.101:3000
curl: (56) Recv failure: Connection was reset

So i don’t understand why with swarm access is KO


(Etienc) #5

You specified a port in the compose file, but not a port mapping. It should be in the format <host-port>:<service-port>, similar to how you did it with the manual docker run... command.

If you don’t specify a port mapping, swarm will assign a random high port, in your case 30002.

Honestly, I’ve never used HAproxy, so I’m not sure what’s happening inside the container, and I can’t advise you on the config. However, I’d try what I suggested above and use curl or telnet to determine if the 3000 —> 3000 mapping works.

Also, tail the service logs to see if there are any clues:

docker service logs -tf <service-id>


(Dahu29) #6

Hi,

I succeed to connect to my database with HAPROXY in using port 80.
I used this compose.yml with these lines

aerospikedb:
    image: aerospike/aerospike-server:latest
    networks:
    - aerospikenetwork
    environment:
      - SERVICE_PORTS=3000
    ports:
      - target: 3000
        published: 3000
        protocol: tcp
        mode: host
    deploy:
        replicas: 4
        endpoint_mode: dnsrr
    labels:
        com.aerospike.description: "This label is for all containers for the Aerospike service"
    command: [ "--config-file","/run/secrets/aerospike.conf"]
    secrets:
    - source: conffile
      target: aerospike.conf
      mode: 0440

But now when i want to scale my service to more than 3 services i get these errors :

$ docker service scale aerospike_aerospikedb=6
aerospike_aerospikedb scaled to 6
overall progress: 3 out of 6 tasks
1/6: running [==================================================>]
2/6: running [==================================================>]
3/6: running [==================================================>]
4/6: no suitable node (host-mode port already in use on 3 nodes)
5/6: no suitable node (host-mode port already in use on 3 nodes)
6/6: no suitable node (host-mode port already in use on 3 nodes)
Operation continuing in background.
Use docker service ps aerospike_aerospikedb to check progress.

What is the problem when there is more than 3 nodes ?
It’s because i have only 3 nodes on my cluster ?