Hi there, so while trying to set up a compose for my app, I realized that docker-compose interprets the fluentd-address setting in logging as being the address of a Fluentd/fluent-bit on the host network, not the compose network; and I don’t want that.
The way I have my infra set up, I want a fluent-bit for every compose that parses and filters and labels an app’s logs, so that my main fluentd can focus only on storing the data.
Is there a way to make compose use an in-compose fluent bit as the fluentd-address?
For now, I just added a port binding from my fluent-bit to a localhost port on the host network, but this isn’t really clean and I don’t want to pollute the ports on my VPS.
Docker compose is just a user interface to the docker api. So what you ask for is not compose scoped, it’s a general docker question.
The daemon must be able to reach the fluentd-address before it starts the container. It doesn’t matter whether it is run as native or containerized process, it matters that it already exists and is reachable when the container - that should use the fluentd log-driver - is created.
Ah, that does make sense, but does Docker use its own container-name DNS for the fluentd-address so that I don’t have to pollute the localhost ports?
(Sure, if I’m exhausting all 65k ports I clearly should be running a k8s cluster or something bigger, but it’s more about having less localhost port bindings to remember)
I would assume the Docker daemon, which is running on host, is managing the logging. So you probably need a host port to work with, not only a port inside a Docker container.