Docker Community Forums

Share and learn in the Docker community.

Setting up Networks on AWS for 3 services - compose & swarm

dockercloud

(Bzmw) #1

I’ve been bashing my head for about a week trying to figure out how to correctly setup the networks in my compose file for my 3 services but I’ve realized that I simply lack the experience and training to set them up correctly. I would love for some guidance.

As a temporary hack for now I’ve just set them all to the host network on a single Manager node and opened all the ports I need on AWS (yeah, 'tis bad.)


So I have 3 services, that I’m deploying through a version 3.4 compose file, into a swarm launched from cloud .docker.com into AWS (central region if that matters).

#1. Custom NodeJS App, built from node/carbon-slim:latest

  • Access from the outside world through port 8899 (secure websocket)
  • Needs communication to the Kurento Media Service on port 8888 8443 (secure websocket)
  • Needs to discover Kurento Media Service through service discover wss://kurento-media-service:8443

#2. Kurento Media Service, built from the phusion/baseimage:latest image.

  • Needs to communicate to the turnserver through port 3478/tcp 3478/udp (perhaps 49152-65535/udp as well not too sure yet)
  • Needs to discover turnserver through service discovery stun:turnserver:3478, turn:turnserver:3478
  • Needs to communicate back to my Custom NodeJS App on port 8888 8443 (websocket)
  • Does NOT need access to the outside world

#3. turnserver (coturn project), built from the phusion/baseimage:latest image.


I am also using Route53 to point an A Record Alias record to the Load Balancer that is launched, this allows me to configure other apps to use my-domain.com:8899 to connect to my Custom NodeJS App, and my-domain.com:3478 to connect to the turnserver.

Where I’m confused and need help is specifically with the networks, and scaling:

#1. I can see that I need either a bridge or overlay network between My NodeJS App, and Kurento Media Service, and perhaps it should also extend over the turnserver?

#2. What kind of network do I need for the turnserver?

#3. What command can I run on launch of a turnserver to properly configure the external-ip, and relay-ip. Right now I’m using ip route|awk '/default/ { print $3 }' for the private ip, and curl http://icanhazip.com for public up.

#4. How does all of this work when I replicate the services (with most concerns around turnserver) Do I need a 3rd party load balancer like HAproxy? If so any suggestions for images?

PS. What’s with the 2 link max for newbies? And number styling is broken.


(Bzmw) #2

To self close this issue.

We made a fatal assumption about the coturn service which caused many of the issues we were having.
We assumed that the TURN server needed to be located behind the NAT within the docker network, but it doesn’t necessarily need to be, there is an option for that provided by coturn but you don’t need to make use of it.

The TURN service is specifically designed to get around a client’s NAT/Firewall to a single peer. When deploying to AWS a docker service appears as a single peer so if you have your TURN server located somewhere else that is fine.

What we ended up doing was removing the coturn service from our stack and creating an AMI for that dockerfile and deploying it separately from our stack.
Sadly we can’t use docker swarm with coturn and AWS because AWS load balancer don’t manage UDP traffic which is necessary for the coturn service.