Aggregate container logs with fluentd running in a container

Docker Engine General
1

Hi all,

I want to aggregate container logs with fluentd running in a container. To use the fluentd log driver automatically I start de docker daemon with the option

--log-driver=fluentd

Which works fine.

The problem is: I don’t have all the logs. I am missing the logs from before fluentd has initialized. After that everything is logged through fluentd. Which was surprising to me, since in the documentation it states that de default maximum size of the log buffer is 8M, so I assumed the docker daemon would buffer all output until it could connect to fluent.

I then tried to start the docker daemon with the following options:

--log-driver=fluentd --log-opt fluentd-async-connect

but now the docker daemon refuses tot start:

FATA[0000] Failed to set log opts: unknown log opt 'fluentd-async-connect' for fluentd log driver Waiting for /var/run/docker.sock

Is this by design, a bug or a misunderstaning on my part?

Kind regards,

David

2

Try doing --log-opt fluentd-async-connect=true.

3

I already tried that, but gave it another shot (maybe I made a typo). Still te same:

Waiting for /var/run/docker.sock FATA[0000] Failed to set log opts: unknown log opt 'fluentd-async-connect' for fluentd log driver

4

What version of docker are you running, and what is the entire, exact command you are running?

5

# docker -v Docker version 1.10.3, build 20f81dd

# docker daemon --log-driver=fluentd INFO[0000] [graphdriver] using prior storage driver "aufs" INFO[0000] Graph migration to content-addressability took 0.00 seconds INFO[0000] Firewalld running: false INFO[0000] Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address INFO[0000] Loading containers: start. INFO[0000] Loading containers: done. INFO[0000] Daemon has completed initialization INFO[0000] Docker daemon INFO[0000] API listen on /var/run/docker.sock

# docker daemon --log-driver=fluentd --log-opt fluent-async-connect=true FATA[0000] Failed to set log opts: unknown log opt 'fluent-async-connect' for fluentd log driver

6

[Docker Forums] [PM] Akismet has temporarily hidden your post :innocent:

7

it looks like in this example, you have fluent-async-connect=true instead of fluentd-async-connect=true

8

:blush:

# docker daemon --log-driver=fluentd --log-opt fluentd-async-connect=true FATA[0000] Failed to set log opts: unknown log opt 'fluentd-async-connect' for fluentd log driver

9

Hmmm. Stumbled upon the changelog for docker 1.11:

Logging

  • …
  • The fluentd log driver learned the following options: fluentd-address, fluentd-buffer-limit, fluentd-retry-wait, fluentd-max-retries and fluentd-async-connect (#19439)

Is the documentation ahead of the release?

10

Ow wait, it was released 21 hours ago…

11

# docker -v Docker version 1.11.0, build 4dc5990

# docker daemon --log-driver=fluentd --log-opt fluentd-async-connect=true INFO[0000] New containerd process, pid: 6988 WARN[0000] containerd: low RLIMIT_NOFILE changing to max current=1024 max=4096 INFO[0001] [graphdriver] using prior storage driver "aufs" INFO[0001] Graph migration to content-addressability took 0.01 seconds INFO[0001] Firewalld running: false INFO[0001] Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address WARN[0001] mountpoint for pids not found INFO[0001] Loading containers: start. INFO[0001] Loading containers: done. INFO[0001] Daemon has completed initialization INFO[0001] Docker daemon commit=4dc5990 graphdriver=aufs version=1.11.0 INFO[0001] API listen on /var/run/docker.sock

:smiley:

13

Read the below article, on how to achieve it easily on Docker using fluentd