Docker compose networking bug

General
,
1

[docker-compose question]

hello all! I’ve been stuck on this for a while so hopefully we can debug together.

I’m using docker compose to bring up three separate services.
Everything builds and comes up great. Health check for the app passes, but I can’t seem to hit it from the host.

I’ve tried the following values for app.ports:

“127.0.0.1:3000:3000”
“3000:3000”
“0.0.0.0:3000:3000”

I’ve also tried to run this with a “host” network, but that also didn’t seem to work and I don’t prefer it because apparently that is not supported on Mac and my local developer environment is Macosx. The prod server is ubuntu.

And I’ve tried defining the default bridge netowrk explicitly:

networks:
default:
driver: bridge

version: "2.4"
services:
    rabbitmq:
        image: rabbitmq
        volumes:
            - ${ML_FILE_PATH}/taskqueue/config/:/etc/rabbitmq/
        environment:
            LC_ALL: "C.UTF-8"
            LANG: "C.UTF-8"
    celery-worker:
        image: ${ML_IMAGE_NAME}
        entrypoint: "celery --broker='amqp://<user>:<password>@rabbitmq:5672//' -A taskqueue.celeryapp worker --uid 1111"
        runtime: ${RUNTIME}  ## either "runc" if running locally on debug mode or "nvidia" on production with multi processors
        volumes:
            - ${ML_FILE_PATH}:/host
        depends_on:
            - rabbitmq
            - app
        environment:
            LC_ALL: "C.UTF-8"
            LANG: "C.UTF-8"
            MPLCONFIGDIR: /host/tmp
        volumes:
            - ${ML_FILE_PATH}:/host
    celery-beat:
        image: ${ML_IMAGE_NAME}
        entrypoint: "celery --broker='amqp://<user>:<password>@rabbitmq:5672//' -A taskqueue.celeryapp beat --uid 1111"
        runtime: ${RUNTIME}  ## either "runc" if running locally on debug mode or "nvidia" on production with multi processors
        depends_on:
            - rabbitmq
            - app
        environment:
            LC_ALL: "C.UTF-8"
            LANG: "C.UTF-8"
            MPLCONFIGDIR: /host/tmp
        volumes:
            - ${ML_FILE_PATH}:/host
    app:
        build: .
        entrypoint: ${ML_ENTRYPOINT}  # just starts a flask app
        image: ${ML_IMAGE_NAME}
        ports:
            - "3000:3000"
        expose:
            - "3000"
        volumes:
            - ${ML_FILE_PATH}:/host
        restart: always
        runtime: ${RUNTIME}
        healthcheck:
            test: ["CMD", "curl", "http:/localhost:3000/?requestType=health-check"]
            start_period: 30s
            interval: 30s
            timeout: 5s
        environment:
            SCHEDULER: "off"
            TZ: "UTC"
            LC_ALL: "C.UTF-8"
            LANG: "C.UTF-8"

Thanks so much for any help!

2

The app container should be reachable by the ip/hostname of the docker engine.

The “expose” declation in the compose file made sense when “container linking” still was a thing. As a compose deployment always provides it’s own network, you can leverage the dns-based service discovery instead. Container linking is considered legacy. You can savely remove it.

You will definitly want to have the “expose” instruction in your Dockerfile though, as it a) documents the port a user can expect a service to be reached and b) allows some management ui’s to prefil the port in the container side of port mapping dialog.

1 Like
3

What’s the ip/hostname of the docker engine? Is that something I have to configure?

4

The ip or hostname of the host that runs the docker engine.

5

Oh I see. That part isn’t working. And one layer deeper, from the the host running the docker engine, I would expect to be able to hit it with curl --location --request GET 'http://localhost:3000/health' and that’s also not working. No response and no logs within the container.

6

Almost seems like you applications inside the container binds to 127.0.0.1 instead of 0.0.0.0, as this would explain why the health check passes, but your application is not accessible from outside. Localhost in the container != localhost on the host

7

Output from docker inspect <CONTAINER>

"Ports": {
                "3000/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "3000"
                    }
                ]
}

That seems to indicate it’s configured as expected, right?

And I tried mapping “127.0.0.1:3000:3000” explicitly (so localhost of host) and on curl --location --request GET 'http://localhost:3000/health' and curl --location --request GET 'http://127.0.0.1:3000/health' I still get no response.

8

The docker inspect output says nothing about whether the application inside the container binds 0.0.0.0 as well or only 127.0.0.1. Since your health check passes, it must be one of both, and it seems like the later is the case.

1 Like
9

omg you solved it!!!

I needed to run my flask app as an “externally visible server”..

Docker compose was working fine the whole time. Thank you!!