Unable to setup Mercure with docker and Nginx (502 Bad Gateway)

Hi!

I’m currently trying to set up a project with Mercure. My initial project has been setup by someone else. I have very few knowledge of either Nginx or Docker.

We’ve tried some solutions found when looking through it but it’s too different from our config to be able to apply any of them… And we don’t want to break anything of course.

That said here is the issue : We have integrated Mercure to our Symfony project and it’s working fine in local environment. However, we can’t make it work in a production environment (meaning HTTPS is enabled). At first, we had issues with cors_policy, but now we just run through a 502 error.

Here is our configuration : docker-compose.yml (mercure part) :

  dj_mercure:
    image: dunglas/mercure
    container_name: dj_mercure
    restart: unless-stopped
    ports:
      - "3000:3000"
    networks:
      - core
    environment:
      SERVER_NAME: ':3000'
      MERCURE_PUBLISHER_JWT_KEY: '${MERCURE_SECRET}'
      MERCURE_SUBSCRIBER_JWT_KEY: '${MERCURE_SECRET}'
      # Set the URL of your Symfony project (without trailing slash!) as value of the cors_origins directive
      MERCURE_EXTRA_DIRECTIVES: |
        cors_origins https://${HOST}
        use_forwarded_headers "1"
    # Comment the following line to disable the development mode
    #command: /usr/bin/caddy run -config /etc/caddy/Caddyfile.dev
    volumes:
      - caddy_data:/data
      - caddy_config:/config

networks:
  core:
    driver: bridge

volumes:
  caddy_data:
  caddy_config:

I’d like to point out that our env variable HOST contains dev.mysite.fr And here is the default.conf file of Nginx :

server {
    listen 80 default_server;

    server_name localhost;
    root /var/www/html/public;
    index index.php index.html index.htm;

    location /.well-known/mercure {
        proxy_pass http://127.0.0.1:3000;
        proxy_read_timeout 24h;
        proxy_http_version 1.1;
        proxy_set_header Connection "";

        ## Be sure to set USE_FORWARDED_HEADERS=1 to allow the hub to use those headers ##
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location / {
        # try to serve file directly, fallback to index.php
        try_files $uri /index.php$is_args$args;
    }

    # Dev
    location ~ ^/(index_dev|config)\.php(/|$) {
        fastcgi_pass dj_php:9000;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        fastcgi_param DOCUMENT_ROOT $realpath_root;
    }

    # Prod
    location ~ ^/index\.php(/|$) {
        fastcgi_pass dj_php:9000;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        fastcgi_param DOCUMENT_ROOT $realpath_root;
        internal;
    }

    # return 404 for all other php files not matching the front controller
    # this prevents access to other php files you don't want to be accessible.
    location ~ \.php$ {
        return 404;
    }

   # return 404 for all other php files not matching the front controller
   # this prevents access to other php files you don't want to be accessible.
   location ~ \.php$ {
     return 404;
   }

}

I only added the location /.well-known/mercure part to the file. Everything else was already set up, so I’ not sure if I should modify it or not.

Finally, we have this code in our Javascript :

const url = new URL('https://dev.mysite.fr/.well-known/mercure');
url.searchParams.append('topic', "http://somesite.local/xyz");
eventSource = new EventSource(url);
eventSource.onmessage = event => { // ...
}

But I keep getting a 502 Bad Gateway error basically saying :

2022/07/25 13:46:59 [error] 44#44: *24 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.5, server: localhost, request: "GET /.well-known/mercure?topic=http%3A%2F%2Fsomesite.local%2Fxyz HTTP/1.1", upstream: "http://127.0.0.1:3000/.well-known/mercure?topic=http%3A%2F%2Fsomesite.local%2Fxyz", host: "dev.mysite.fr", referrer: "https://dev.mysite.fr/fr/admin/dashboard"

We feel like we’re really close to the answer, but our lack of knowledge is getting in the way. We have not a complete idea of what we’re doing, so is there anyone here who could help us understand and fix this ?

Thanks!

This would mean the target service is running in the same nginx container on port 3000? Is it the case?

Note: localhost in the container is neither the same localhost of the host, nor the same localhost of another container.

Thanks a lot for your quick answer.

The answer is : no, the target service (Mercure hub) is running on a separate, dedicated container, on port 3000.

Should I change something consequently ?

Yes, but it depends on how nginx is run: also as a container?

Yes, correct, Nginx is also run as a separate container.

Here is an excerpt from docker-compose.yml file regarding nginx container :

  dj_nginx:
   image: "mdg/djnginx:1.19.9"
   container_name: dj_nginx
   volumes:
     - /etc/localtime:/etc/localtime:ro
     - /etc/timezone:/etc/timezone:ro
   networks:
     - core
   depends_on:
      - dj_mariadb
      - dj_php
   volumes:
     - ./symfony/:/var/www/html
   labels:
     - "traefik.enable=true"
     - "traefik.http.routers.mdg-front-http.rule=Host(`${HOST}`)"
     - "traefik.http.routers.mdg-front-http.entrypoints=web"

Please paste code as Preformated text block (</> icon).

Please also share the network declaration, so I can see what you actualy do there.

What you want to do is to create a networks from the cli and use it as external network in both the nginx compose file and the other services compose file. Then you can leverage the build-in dns-based service discovery, which allows to address a container to access another container by it’s service name.

version: '3.8'

services:
  dj_traefik:
   image: traefik:2.4.12
   restart: unless-stopped
   command: >
            --global.checknewversion=true --global.sendanonymoususage=false
            --entrypoints.name=web --entrypoints.web.address=":80"
            --log=true --log.filepath=/log/traefik.log --log.format=common --log.level=INFO
            --accesslog=true --accesslog.filepath=/log/access.log --accesslog.format=common
            --api --api.insecure=true
            --providers.file.directory=/conf/ --providers.file.watch=true
            --providers.docker=true --providers.docker.exposedbydefault=false --providers.docker.swarmmode=false
            --certificatesresolvers.myresolver=true
            --certificatesresolvers.myresolver.acme.email="admins@<redacted>"
            --certificatesresolvers.myresolver.acme.storage="acme.json"
            --certificatesresolvers.myresolver.acme.keytype="EC384"
            --certificatesresolvers.myresolver.acme.httpchallenge=true
            --certificatesresolvers.myresolver.acme.httpchallenge.entrypoint=web
   container_name: dj_traefik
   networks:
     - core
   ports:
     - "80:80"
     - "8080:8080"
   volumes:
     - ./docker/djtraefik/acme.json:/acme.json
     - ./docker/djtraefik/conf:/conf
     - ./log/traefik:/log
     - /etc/localtime:/etc/localtime:ro
     - /etc/timezone:/etc/timezone:ro
     - /var/run/docker.sock:/var/run/docker.sock
  #   - /run/docker.sock:/var/run/docker.sock:ro
   labels:
     - "traefik.enable=true"
     - "traefik.http.routers.api.rule=Host(`proxy.localhost`)"
     - "traefik.http.routers.api.service=api@internal"

  dj_php:
   image: "mdg/djphp:8.1.1-fpm-buster"
   container_name: dj_php
   volumes:
     - /etc/localtime:/etc/localtime:ro
     - /etc/timezone:/etc/timezone:ro
   networks:
     - core
   volumes:
     - ./symfony/:/var/www/html

  dj_nginx:
   image: "mdg/djnginx:1.19.9"
   container_name: dj_nginx
   volumes:
     - /etc/localtime:/etc/localtime:ro
     - /etc/timezone:/etc/timezone:ro
   networks:
     - core
   depends_on:
      - dj_mariadb
      - dj_php
   volumes:
     - ./symfony/:/var/www/html
   labels:
     - "traefik.enable=true"
     - "traefik.http.routers.mdg-front-http.rule=Host(`${HOST}`)"
     - "traefik.http.routers.mdg-front-http.entrypoints=web"

  dj_mariadb:
    image: "mdg/djmariadb:10.5.9"
    container_name: dj_mariadb
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro
    networks:
      - core
    volumes:
      - ./mysql/:/var/lib/mysql
    ports:
      - "3306:3306"
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
      MYSQL_ALLOW_EMPTY_PASSWORD: 'no'

  dj_mercure:
    image: dunglas/mercure
    container_name: dj_mercure
    restart: unless-stopped
    networks:
      - core
    ports:
      - "3000:3000"
    environment:
      SERVER_NAME: ":3000"
      MERCURE_PUBLISHER_JWT_KEY: 'eyJ0eXAiOiJ<redacted>kaXM'
      MERCURE_SUBSCRIBER_JWT_KEY: 'eyJ0eXAiOiJ<redacted>kaXM'
      # Set the URL of your Symfony project (without trailing slash!) as value of the cors_origins directive
      MERCURE_EXTRA_DIRECTIVES: |
        cors_origins http://thefastjourney.local
    # Comment the following line to disable the development mode
    command: /usr/bin/caddy run -config /etc/caddy/Caddyfile.dev
    volumes:
      - mercure_data:/data
      - mercure_config:/config
###< symfony/mercure-bundle ###

networks:
  core:
    driver: bridge

volumes:
###> symfony/mercure-bundle ###
  mercure_data:
  mercure_config:
###< symfony/mercure-bundle ###

This is the full docker-compose.yml file with hopefully the requested information. Thanks a lot.

See, now I know both services are in the same compose file and can access each other by service name.
It would have been better to aonymize credentials and tokens…

You have two possibilites:

  1. use path based rules in traefik and bypass the nginx container for the dj_mercure traffic
  2. keep nginx and use proxy_pass http://dj_mercure:3000; in your nginx.conf instead of proxy_pass http://127.0.0.1:3000;. Since both container share the same network,nginx can access the dj_mercure`service by its service name. )See NGINX swarm redeploy timeouts - #5 by meyay for further optimizations)

Solution 1 is the better solution, as traefik adds/removes reverse proxy rules then a container start or stop event is published.

`

2 Likes

I hope I got them all (I redacted some, and have hidden the first revision).

1 Like

First thanks a lot for your help. We implemented solution 2 (put http://dj_mercure:3000 instead of http://127.0.0.1:3000) and it worked! We still have an issue to resolve : we have to put anonymous in the MERCURE_EXTRA_DIRECTIVES because otherwise we are getting 401 error. So it seems that we still have a problem with JWT to solve. Thanks again.

Make sure to check the link I posted for solution 2 to no be affected by dns caching. By default nginx resolves the target once and caches the ip indefinitly - the link shows how to bypass that. Dns caching is a problem when the target container restarted or was recreated, but the nginx container was not.

Traefik on the other hand is not affected by a dns caching issue at all.

We have solved our question with use of solution 2. We have also solved our issue with JWT. We will also try to implement your solution 1 as you suggest this is the best one.
Thanks.

1 Like