Docker-compose replicas with nginx reverse proxy

For the Example Voting App for docker, I am trying to start multiple replicas of the vote app and dynamically allocate host port to container port 80.

Then, I tried to setup a nginx as a reverse proxy that would forward all requests for vote app to the associated container.

For some reason, when I hit the URL for the ex2 instance, I continue to get the nginx default page and not the voting page.

docker-compose.yml

   services:
  vote:
    image: ritikbilala/dockervotingapp_vote:latest
    deploy:
      replicas: 2
    # use python rather than gunicorn for local dev
    command: python app.py
    depends_on:
      redis:
        condition: service_healthy
    volumes:
     - ./vote:/app
    networks:
      - front-tier
      - back-tier
  rproxy:
    build: ./rproxy
    deploy:
      replicas: 1
    depends_on:
      - vote
    ports:
      - "80:80"
    networks:
      - front-tier
  result:
    image: ritikbilala/dockervotingapp_result:latest
    deploy:
      replicas: 1
    # use nodemon rather than node for local dev
    command: nodemon server.js
    depends_on:
      db:
        condition: service_healthy
    volumes:
      - ./result:/app
    ports:
      - "8080:80"
      - "5858:5858"
    networks:
      - front-tier
      - back-tier

  worker:
    image: ritikbilala/dockervotingapp_worker:latest
    deploy:
      replicas: 1
    depends_on:
      redis:
        condition: service_healthy
      db:
        condition: service_healthy
    networks:
      - back-tier

  redis:
    image: redis:5.0-alpine3.10
    deploy:
      replicas: 1
    volumes:
      - "./healthchecks:/healthchecks"
    healthcheck:
      test: /healthchecks/redis.sh
      interval: "5s"
    ports: ["6379"]
    networks:
      - back-tier

  db:
    image: postgres:9.4
    deploy:
      replicas: 1
    environment:
      POSTGRES_USER: "postgres"
      POSTGRES_PASSWORD: "postgres"
    volumes:
      - "db-data:/var/lib/postgresql/data"
      - "./healthchecks:/healthchecks"
    healthcheck:
      test: /healthchecks/postgres.sh
      interval: "5s"
    networks:
      - back-tier

volumes:
  db-data:

networks:
  front-tier:
  back-tier:

Dockerfile for nginx

    # Using latest ubuntu base image
    FROM ubuntu:latest
    
    LABEL author="Debal Das"
    
    # add curl for healthcheck
    RUN apt-get update \
        && apt-get install -y nginx
    
    
    # Copy reverse proxy configuration file
    COPY default.conf /etc/nginx/conf.d/default.conf
    COPY ./includes/ /etc/nginx/includes/
    #RUN nginx -t
    # Make port 80 available for links and/or publish
    EXPOSE 80
    
    # Define our command to be run when launching the container
    CMD ["nginx", "-g", "daemon off;"]

default.conf

    server {
            listen 80;
            listen [::]:80;
            server_name _;
            location / {
                proxy_pass http://vote:80;
            }
        }

Output from running curl vote:80 inside the nginx container

root@258365be8730:/# curl vote:80
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>Cats vs Dogs!</title>
    <base href="/index.html">
    <meta name = "viewport" content = "width=device-width, initial-scale = 1.0">
    <meta name="keywords" content="docker-compose, docker, stack">
    <meta name="author" content="Tutum dev team">
    <link rel='stylesheet' href="/static/stylesheets/style.css" />
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.4.0/css/font-awesome.min.css">
  </head>
  <body>
    <div id="content-container">
      <div id="content-container-center">
        <h3>Cats vs Dogs!</h3>
        <form id="choice" name='form' method="POST" action="/">
          <button id="a" type="submit" name="vote" class="a" value="a">Cats</button>
          <button id="b" type="submit" name="vote" class="b" value="b">Dogs</button>
        </form>
        <div id="tip">
          (Tip: you can change your vote)
        </div>
        <div id="hostname">
          Processed by container ID aff76a269759
        </div>
      </div>
    </div>
    <script src="http://code.jquery.com/jquery-latest.min.js" type="text/javascript"></script>
    <script src="//cdnjs.cloudflare.com/ajax/libs/jquery-cookie/1.4.1/jquery.cookie.js"></script>


  </body>

You don’t need to define the ports on the vote service, as the rproxy service is the one that handles the incomming requests. Traffic between rproxy and vote use a container network, which is not restricted in any way.

Also, there is no need to share all networks between the vote and rproxy service. It is sufficient to share a a single network between rproxy and target services. If you just use front-tier network it will be fine.

When it commes to you nginx.conf. make sure you read and follow the suggestions from here:

The problem and the fix is the same, regardless wether you deploy swarm stacks or docker-compose stacks.

I tried following the solution and have updated the default.conf

server {
        listen 80;
        listen [::]:80;
        resolver 127.0.0.11 valid=10s;
        resolver_timeout 5s;
        server_name _;
        location / {
            set $target http://vote:80;
            proxy_pass $target;
        }
    }

I continue to face the same problem. I noticed, I am able to curl the http://vote:80 from within the container. So that kind of rules out network issues. And like you said, must be related to the nginx conf.

In the nginx container, I haven’t made any changes to any other file nor have I created any symlinks. Am I missing any of the steps?

Replicas follow to “cattle” philosophy, not the “pets” philosopy. Thus said, I am unsure what “URL for the ex2 instance” would mean?

Furthermore your vote service depends_on the redis service to become healty. I assume you removed it from the compose file you pasted, as otherwise it would not even make sense that any of the vote replicas is even reachable by the RP.

I’ve updated the complete docker-compose.yml

I have launched the containers in a AWS EC2 instance and am trying to hit the vote application using ec2ipaddress:80

nginx is listening on port 80 on the host machine.

Public ip? Private Ip? Does the security group of the compute instance allow it?

Yep. The correct ports are open.

So, I’ve tried to build the image differently this time -

FROM nginx

LABEL author="Debal Das"

RUN rm /etc/nginx/conf.d/*
COPY default.conf /etc/nginx/conf.d/

default.conf

server {

  listen 80;

  location / {
    proxy_pass http://vote:80;
  }
}

And this seems to work like a charm. Am I missing any steps when I tried to install nginx on a ubuntu image and configure it manually?

Instructions followed from 6. Understand Docker Compose Basic By Creating Nginx Reverse Proxy - YouTube

Well, I used it with a nginx.conf according the link I send earlier on EC2 instances and it worked like a charm… It should have worked for you as well.

I don’t see why it should not work for you, but appearently it does not work.

Forgive me if I am not going to look at that video tutorial.

It should work, if all this is true:

  • the security group of the ec2 instance is configured correctly
    • If it’s for a public ip: ingres to port 80 tcp from 0.0.0.0
    • If it’s for a private ip: ingres to port 80 tcp from your vpc’s az’s
  • you installed docker following the instructions from docker’s documentation and actualy installed docker from docker’s repositories
  • Your compose file and your nginx.conf look like you shared them
    • N.B.: your first and last version of the nginx.conf will be prone to dns caching.

I am out of ideas. I leave it for someone else to answer.

I really appreciate the tips you’ve provided. Am new to docker and trying to implement practice projects to get a better feel for it. I will play around with the version you’ve suggested and see if I can figure something out. I have a strong feeling I am missing something small which after I identify will look like a complete bufoon.

Cheers.