Container in network_mode:host using depends_on linked/service:host-container

[ffaabbzz]

I have a container that needs network_mode: host because of clientIP’s that I need to be able to get and not always the XFF headers are there.

That same container needs to connect to a backend that needs to be up first and uses a healthcheck for that.

Issue that happens is that I cannot put a container in network_mode:host to another network so I set network_mode: service:host-container

When I do so I cannot set a depends_on: backend-container the service:host-container already sets some depends_on and I get the following:

dependency cycle detected: frontend -> backendl -> frontend

Is there still a way for docker to let frontend wait to start after backend is online in this case ?

[meyay (Metin Y.)]

Could you share a minimal compose file that shows your configuration.

I am inconclusive if network_mode: host is still relevant to your post or not. As a result a container would use the host network and has not network isolation at all. Means the container can not be part of other isolated networks.

If a container is using network_mode: service:host-container it will join the network namespace of the host-container service - as such the host-container must be started for the joining container to succeed.

@ffaabbzz

When I do so I cannot set a depends_on: backend-container the service:host-container already sets some depends_on and I get the following:

dependency cycle detected: frontend -> backendl -> frontend

Can you rephrase that, I don’t really understand the issue. Though, your host-container service can not depend on a service that uses network_mode: service:host-container, as you would have cyclic dependencies that would never be able to be satisfied.

[meyay (Metin Y.)]

Recent docker-ce versions should retain the host ip. If you deploy this compose project:

services:
  whoami:
    image: traefik/whoami
    ports: ["80:80"]

With new docker version (I tested 28.1.1 and 29.7.2) You can see that the RemoteAddr retains the client-ip and the outgoing tcp port.

me@mac \~ % curl <docker-host-ip>
Hostname: e091f737ddc0
IP: 127.0.0.1
IP: ::1
IP: 172.18.0.2
RemoteAddr: <client-ip>:65524
GET / HTTP/1.1
Host:<docker-host-ip>
User-Agent: curl/8.7.1
Accept: \*/\*

With older docker engines (I tested v24), the RemoteAddr shows the docker bridge network’s gateway ip (something like 172.18.0.1:65524) the container is connected to , instead of the client-ip and offcourse the random tcp port. I am not sure in which exact docker version his behaviour changed.

Note: the behavior for Docker Desktop is different, as with new versions the RemoteAddr will show the ip of the utility vm, that runs the actual Docker Engine of Docker Desktop.

[ffaabbzz]

I think this makes what I stated more clear; I think you need to re-ask after seeing this ?

services:
  frontend:
    image: frontend-image
    restart: unless-stopped
    network_mode: host # needed for hostIP usage without any assinging in the dockerfile
    depends_on:
      backend:
        condition: service_healthy

  backend:
    image: backend-image
    restart: unless-stopped
    network_mode: "service:frontend"
#    links: // same issue as depends_on for service: frontend
#      - frontend
    healthcheck:
      test: [ "CMD", "healthcheck.sh", "--connect", "--innodb_initialized" ]
      start_period: 1m
      start_interval: 10s
      interval: 1m
      timeout: 5s
      retries: 3```

[meyay (Metin Y.)]

You try to configure a cyclic dependency. Let’s assume for a minute that frontend uses a bridge network, it is supposed to start once backend is healthy. The backend service can not start before the frontend service is started, as it provides the network namespace the backend service is configured to use.

I am not sure if network_mode: "service:frontend" is possible if fronted uses network_mode: host. I would be surprised if it did.

Though, why wouldn’t you configure network_mode: host for the backend service as well? After all it will result in using the host networking without isolation.

Generally, with recent docker-ce versions, it shouldn’t be necessary to use network-host just to retain the source-ip. Unless, you are on Docker Desktop, then the setting enable host networking in combination with network_mode: host would be your only chance to retain the client ip.

[ffaabbzz]

@meyay

You try to configure a cyclic dependency. Let’s assume for a minute that frontend uses a bridge network, it is supposed to start once backend is healthy. The backend service can not start before the frontend service is started, as it provides the network namespace the backend service is configured to use.

Yes that is known: I’m bound to what I use so we can discuss whatever we want but this has all been tested; I only care about the delay start based on service using docker itself.

@meyay

I am not sure if network_mode: "service:frontend" is possible if fronted uses network_mode: host. I would be surprised if it did.

Yes it does, on 127.0.0.1 you can connect perfectly fine; that is also in the docs for that backend networkmode and ‘hostmode’ cannot connect to any other network anyways; so I think there is no unseen gap there.

@meyay

Though, why wouldn’t you configure network_mode: host for the backend service as well? After all it will result in using the host networking without isolation.

I don’t want backend in any case connected to the host interface directly; nope not even firewalled.

@meyay

Generally, with recent docker-ce versions, it shouldn’t be necessary to use network-host just to retain the source-ip. Unless, you are on Docker Desktop, then the setting enable host networking in combination with network_mode: host would be your only chance to retain the client ip.

I see the opposite about Docker desktop; but even because of the hostIP I want to use without any hardcoded IP in the docker-compose.yml I’m bound to ‘host’ anyways.

Can you point me to the docs about which states what you say ? I like to see for othersetups with caddy where I still publish with mode: host

[meyay (Metin Y.)]

Given your answer is it safe to assume that you use Docker Desktop:

@ffaabbzz

Yes it does,

Good to know.

@ffaabbzz

I don’t want backend in any case connected to the host interface directly; nope not even firewalled.

You join the backend service to the network of the frontend service, which itself joined host networking. Thus, both services and host networking use the same non-isolated host network, so of course all host ip’s , including localhost will be identical for both services and the host.

So if you want the wait to work properly, I don’t see a way around using network_mode: host for backend as well. It changes nothing on how the backend is attached the host networking. Network-wise result is the same.
But now you can use depends on the way you want.

Note: with Docker Desktop host networking means the host networking of the utility vm, not your real host. Docker Desktop does “magic” on the network layer to mirror packages between host and utility vm in both directions so they appear like they come from the same host. Technically sharing the real host interfaces between host and utility vm is impossible.

Even though the Docker Desktop host is not able to reach container networks directly, a container running with host networking, is able to reach whatever the utility vm reaches (like docker-ce would do). A docker-ce host is able to reach every containerized service using its container-ip:container-port.

@ffaabbzz

I see the opposite about Docker desktop; but even because of the hostIP I want to use without any hardcoded IP in the docker-compose.yml I’m bound to ‘host’ anyways.

Can you rephrase that?

@ffaabbzz

Can you point me to the docs about which states what you say ? I like to see for othersetups with caddy where I still publish with mode: host

I can’t. I would need to google it, so I will leave it to you. If I was able to pinpoint in the docs when the behaviour changed, I would have mentioned the exact version where this behaviour change was introduced.

When it comes to the results I shared: I tested it using the compose file I shared. All tests are done with docker-ce in Linux VMs, except the test with Docker Desktop.

Note: depenods_on is a nice idea if your target environment is a docker engine. But if your production environment is Kubernetes, Docker Swarm or AWS ECS, then you will need to find another solution, as depends on doesn’t exist there. The clean solution would be to make the frontend service resilient against unavailability of the database, regardless if it’s during start or during runtime.

Update: I did my test for publishing bridge networks with curl hostip. It seems if I test it with curl localhost on the docker-ce host directly, the client ip is not retained, and the docker network’s gateway ip is shown as client-ip.

[ffaabbzz]

@meyay

Given your answer is it safe to assume that you use Docker Desktop:

Nope just Docker-CE on Ubuntu (server); wrong assumtion I cannot place.

You join the backend service to the network of the frontend service, which itself joined host networking. Thus, both services and host networking use the same non-isolated host network, so of course all host ip’s , including localhost will be identical for both services and the host.

Yes now you say and I test it with a firewall rule to the backend on the host interface that indeed is able to be connect;ed which I don’t like much. It wasn’t clear and as I automated my firewall rules that backend server won’t ever be opened by the firewall but I would love to have it isolated as I do with most networks internal/external.

So if you want the wait to work properly, I don’t see a way around using network_mode: host for backend as well. It changes nothing on how the backend is attached the host networking. Network-wise result is the same.
But now you can use depends on the way you want.

Yes so both network_mode: host might be the only option and set the backend accept localhost only connections.

Note: with Docker Desktop host networking means the host networking of the utility vm, not your real host. Docker Desktop does “magic” on the network layer to mirror packages between host and utility vm in both directions so they appear like they come from the same host. Technically sharing the real host interfaces between host and utility vm is impossible.

Yes one of the reasons I stopped using Desktop back the days

Can you rephrase that?

No need, I read what you said about people having those issues on Desktop

I can’t. I would need to google it, so I will leave it to you. If I was able to pinpoint in the docs when the behaviour changed, I would have mentioned the exact version where this behaviour change was introduced.

OK, yes everything is hard to find all the time going over the years.

Note: depenods_on is a nice idea if your target environment is a docker engine. But if your production environment is Kubernetes, Docker Swarm or AWS ECS, then you will need to find another solution, as depends on doesn’t exist there. The clean solution would be to make the frontend service resilient against unavailability of the database, regardless if it’s during start or during runtime.

Yes I design compose files per environment; no worries.

Thanks!