Interconnecting services in 1.12 swarm mode (Nginx in front of Elastic)?

Open Source Projects Swarm
1

I am currently running on Docker 1.12.0-rc4. I am wondering how I can interlink services between each other.

Assuming I have a 2 server Elastic cluster which does not provide authentication, connecting to each other via an overlay network. (Note: there may be still a bug in rc4 which prevent this from working correctly)
docker service create \ --name elastic \ --replicas=2 \ --publish 9200 \ --publish 9300 \ --network NetworkElastic \ elasticsearch \ -Ddiscovery.zen.ping.unicast.hosts=elastic
Now I want to put a proxy on each server in front of Elastic to enable authentication
docker service create \ --name proxy9200 \ --replicas=2 \ --publish 9200:80/tcp \ --network NetworkElastic \ --env PASSTHRU_DESTINATION=http://elastic:9200 \ --env PASSTHRU_USERNAME=elastic \ --env "PASSTHRU_PASSWORD=my-pass" \ collinsongroup/nginxpasswordproxy
How would I configure the nginx container to forward to elastic service port 9200? Best case to stay on the same server to not generate too much network traffic. Parameter --link does not exist for services.

I think the swarm create documentation could be expanded a bit about this.

2

Yes, parameter --link does not exist in service create, you don’t need it anyway. You can use in build service discovery. In your case nginx container should be able to access elastic’s api via http://elastic:9200 and http://elastic:9300. Swarm creates a dns same as name parameter in a specified network. Problem is you can’t ping to prove it, attach /bin/bash to a running container and use curl command to verify it.

3

One thing that you might find (which we did) is that nginx will not start up unless the DNS resolves on startup. What that means is that your container will just keep restarting and restarting until the elastic service is available. What we did instead is change the proxy9200 service to have a CMD in the Dockerfile to wrap the call to start nginx. Here is what we used:

#!/bin/bash

knownError=false
until [[ $knownError == true ]]; do
    results=$(nginx -g 'daemon off;' 2>&1 >/dev/null)

    if [[ "$results" == *"host not found in upstream"* ]]; then
        echo "Results contained the expected error. Going to cycle the process and try again"
        sleep 5s
    else
        knownError=true
    fi
done

echo "=========================================="
echo "An unknown error occurred. See output below:"
echo $results
echo "=========================================="
4

“(Note: there may be still a bug in rc4 which prevent this from working correctly)”

Just for completeness: this same issue affects Crate.io as well. Using Docker Engine 1.12, it currently seems impossible that Crate nodes can communicate with each other over an overlay network.