How to dynamically configure a reverse proxy when using profiles in docker-compose

I am trying to create a local docker-compose environment for development. I might have cornered myself into a dead end after a day’s work on the docker-compose file, or I’m not thinking about something correctly here so I’ve come for thoughts and suggestions.

I have services A, B, C and D.

  1. When developing profile “foo” I want to run A and B.
  2. When developing “bar” I want to run A and C.
  3. Service B and C both “depends_on” D.

Service E is nginx, reverse proxy to A, B and C, and I am trying to configure nginx. However I have run into an issue.

If I do docker-compose up --profile foo, then I don’t want or need to include service C.
If I do docker-compose up --profile bar then I don’t want or need to include service B.

How can I configure nginx such that it only includes the configs for the current profile? I actually have a dozen services, and various profiles that include them in several combinations, and I don’t want to be forced to run 8 or 10 containers I won’t be using most of the time.

The problem is that if I include all the upstream backends for all the services, then nginx complains that [the missing one] isn’t there, and it quits the container. But I can’t figure out how to configure docker-compose and call up and pass a profile and have it omit the configurations I don’t want/need for that profile.

Is this making sense? How can this be achieved?

Hello Geoffrey,

maybe Traefik could help here instead of using NginX as reverse proxy.
Traefik is a reverse-proxy-loadbalancer which you can run as a Docker-container. To add some new revers-proxy-service you add some labels to the containers where the request should be proxied to instead of directly modifying the reverse-proxy’s config. This way you have a reverse proxy and it’s configuration/rules are adapted dynamically according to the labels at the backend-containers.

I moved the topic from “Tips & HowTos” to “Open Source Projects » Compose” since the other category is for sharing tips not asking for them.

Please, share how you configure your Nginx Proxy now. A compose file could help a lot. If you don’t want to share the original compose file, create a smaller example which has the same result.

However Traefik, mentioned by @matthiasradde is popular and it is worth to try, Nginx Proxy can work very similarly unless you configure an Nginx container manually. Instead of that, use this one:

https://hub.docker.com/r/nginxproxy/nginx-proxy

or Traefik of course.

I have been wanting to try Traefik, but the issue is with (my lack of experience with advanced) docker-compose. I am using profiles at the moment, but I don’t think I am approaching the problem correctly. You can

but to achieve what I am after, I needed to use multiple compose files and my own wrapper bash script to decide which docker files to include.

As an example, say I have a legacy site and a new API we’re building. I sometimes only want the new site running, other times I only need the legacy site (bug tickets) and other times I need both running. If I just use the dockerfile in each repo, then port 80 is in conflict inside Docker when I try to bring up the 2nd site, so I need a way to proxy them to get them both running at the same time. I was trying to figure out a way to use profiles or extends in a single docker-compose file, but I need to be able to configure nginx differently depending on the context.

What I ended up doing was splitting up my single docker-compose file into separate pieces…

  • docker-compose.nginx.yml - reverse proxy
  • docker-compose.php7.4.yml - legacy site
  • docker-compose.php8.1.yml - new api
  • docker-compose.mysql5.7.yml - legacy db
  • docker-compose.mysql.8.0.yml - new db
  • docker-compose.react.fe.yml - new frontend
  • docker-compose.network.yml - network and volumes

So my bash script does variations of this:

# run the legacy site
docker-compose \
   -f docker-compose.nginx.yml \
   -f docker-compose.php7.4.yml \
   -f docker-compose.mysql5.7.yml \
   -f docker-compose.network.yml \
   up -d

I actually have to pass a dynamic variable to nginx to flag which configs to use, but I managed to get it working. There is probably a more elegant way to do it, but so far, this was the best I could come up with in a day.

Hello Geoffrey,

if you want to go with traefik I would go for a slightly different layout (as I don’t know what docker-compose.network.yml is doing I skip it in this example):

  • one docker-compose.yml for Traefik (the reverse-proxy/loadbalancer)
  • one docker-compose.yml with its NginX + PHP7.4 + MySQL5.7 for the legacy site
  • one docker-compose.yml with its NginX + PHP8.1 + MySQL8.0 for the new API
  • one docker-compose.yml for the new frontend

The Traefik-container is always startet and listens to :80 on the computer’s outer interfaces.
The other containers are only started if you want/need them to.
And each NginX-/Frontend-container brings its own labels telling Traefik which rules to proccess to forward traffic to this NginX-container.
The NginX-/Frontend-containers do not need to be directly accessible from the outside - only in a docker-internal network shared between Traefik and these NginX-containers (usually the corresponding PHP- and MySQL-containers are in this network, too).
In my opinion Traefik is hard to learn in the first place. But after this learning-time it is so easy and fun to add other webservices/APIs/websites without ever touching Traefik’s config again.

1 Like

Have you checked what I linked?

You can still go with Traefik, but the fact that you still wondering if there is a “more elegant way” makes me think that you did not click the link I shared.

You don’t need to do that if your reverse proxy dynamically generates its configuration file communicating with the Docker API. Whether you choose Nginx Proxy or Traefik is up to you. However if you just try either of them without understanding the concept it will be hard for you to configure it properly, I feel you don’t understand it yet, because despite the title it doesn’t matter whether you use profiles or not.

I don’t want to tell you which one is easier, because I know someone who could not configure Nginx Proxy, but Traefik was easy for him. But most of the people I know finds Nginx Proxy easier. I am still using it, but only becase I didn’t have time to switch to Traefik. It’s in my bucket list, because I liked it when I tried.

Traefik and Nginx Proxy use the docker api and process container start and stop events. Both evaluate the labels on the target container from the event and dynamically add/remove reverse proxy rules based on the configuration from the labels. What could be more beautiful than that? :slight_smile:

I have attempted to switch from Nginx to Traefik and it is not working. I am getting bad gateway from Traefik and even though I am running the most basic of configurations copied identically from the docker-compose basic Traefik example. It is not clear what I need to do.

I have the desire to run multiple services in docker-compose and each Dockerfile/image/service exposes different ports. let’s say four services inside docker each expose their own custom port 801, 802, 803, 804. It would be advantageous to expose 3306 MySQL to the Host, but irrelevant to this particular discussion.

I got this working with nginx, using my-service as an upstream and nginx proxy based on the server name, without issue. With this config in nginx it works as expected…

upstream my.service {
    server my.service:801;
} 

server {
    listen 80;
    server_name my.service;

    location / {
        proxy_pass         http://my.service;
        proxy_redirect     off;
    }
}

Note: I have added my.service, other.service, third.service and fourth.service to /etc/hosts pointing to 127.0.0.1

Again, this works fine in nginx, but I’m trying to solve the issue (with Traefik autodiscovery) so as to not need to dynamically switch configurations if I only want my.service and fourth.service, or other.service and third.service.

So I created a branch, ripped out nginx, and tried replacing it with Traefik, working on the first service only, my.service, that exposes port 801 inside its Dockerfile and its apache site.conf, I don’t need or want port 801 exposed to the host.

Here is my Traefik config in docker-compose:

  traefik:
    image: "traefik:v2.9"
    container_name: "traefik"
    command:
      - "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
    ports:
      - "80:80"
      - "8080:8080"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
    networks:
      - "custom-network"

I added the following to my.service definition in docker-compose:

    networks:
      - "custom-network"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.my-service.rule=Host(`my.service`)"
      - "traefik.http.routers.my-service.entrypoints=web"

I docker-compose up and browse to http://my.service/page.php

  • I can’t seem to understand why, but Traefik just gives a “bad gateway” error.
  • I don’t want to do routing based on external ports, I want to use hostnames.
  • I don’t want to add custom prefixes, either, and with nginx I don’t have to.

I want:

The Traefik interface on 8080 works great and looks amazing. In fact it says successful routing for HTTP and Host(my.service) is listed. It says 3 HTTP services are configured, my.service being one of them.

From the Traefik debug logs:

time="2022-10-19T17:52:00Z" level=debug msg="Creating server 0 http://192.168.160.4:80" serverName=0 serviceName=my-service-folder-name entryPointName=web routerName=my-service@docker
time="2022-10-19T17:52:00Z" level=debug msg="Added outgoing tracing middleware my-service-folder-name" middlewareType=TracingForwarder entryPointName=web routerName=my-service@docker middlewareName=tracing
...
time="2022-10-19T18:02:48Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" ForwardURL="http://192.168.160.4:80" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/favicon.ico\",\"RawPath\":\"\",\"OmitHost\":false,\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8\"],\"Accept-Encoding\":[\"gzip, deflate\"],\"Accept-Language\":[\"en-US,en;q=0.9\"],\"Connection\":[\"keep-alive\"],\"Cookie\":[\"_fbp=fb.1.1665634201114.1532094668; _gid=GA1.2.391617166.1666121711; _ga=GA1.1.993995607.1665634201; _ga_280913663=GS1.1.1666121711.1.0.1666121713.0.0.0\"],\"Referer\":[\"http://my.service/page.php\"],\"User-Agent\":[\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36\"],\"X-Forwarded-Host\":[\"my.service\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"f7e64b0518e1\"],\"X-Real-Ip\":[\"192.168.160.1\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"my.service\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"192.168.160.1:57280\",\"RequestURI\":\"/favicon.ico\",\"TLS\":null}"
...
time="2022-10-19T18:02:48Z" level=debug msg="'502 Bad Gateway' caused by: dial tcp 192.168.160.4:80: connect: connection refused"

folder-name is the name of my directory where the docker-compose file is (aka the cloned repository where my docker environment containing Traefik and the other services.

So somehow Traefik seems to not be getting my service port (801) from configuration correctly.

This seems simple, but I’m not sure what I am missing, if you could give any insight I would be very grateful.

Afaik auto port detection will only work if the port is indeed exported in the Dockerfile while building the image.

It should work for my.service, if you add this additional label to the list AND port 801 indeed is the port the container list listening on:

- traefik.http.services.my-service.loadbalancer.server.port:801

Also you need to make sure Traefik and the target container are in a shared docker network.

After adding this, I get 404 page not found.

Yet, if I add this to my.service:

   expose: 
      - "801:801"

and browse http:// my .service:801/page.php - the page comes up, so I know the backend is working.

The error in the Traefik logs:

time="2022-10-19T20:37:33Z" level=error msg="field not found, node: port:801" providerName=docker container=my-service-folder-name-eacebede1aa78dd7054c116933c551782ae244e606c56b331130ebcd98557e1b

The problem is just a typo

In docker-compose.yml for Traefik, this is needed:

  - "traefik.http.services.my-service.loadbalancer.server.port=801"

Equals, not colon. Works now.

Right, I copied it from an example that used key/value instead of a sequence and forgot to replace the colon with an equal sign.

One more thing: what’s wrong with using the default ports inside the containers? I never change container ports unless there is a real reason to do so.

I was under the impression that you could only have one service per port inside the same docker container but the ports don’t conflict if they’re in separate containers. You are correct, and I am changing back to the standard ports.

Also, Traefik is awesome, you were right.