Docker Community Forums

Share and learn in the Docker community.

UCP 2.2.4 logs to ELK 6.1.0

docker

(Nicolas Bihan) #1

Hi,

I tried to get the UCP logs in my Logstash server (part of ELK) and get an error in logstash

[2017-12-21T05:38:57,812][WARN ][logstash.filters.json    ] Error parsing json {:source=>"message", :raw=>"<7>2017-12-21T05:38:57Z ucp-controller-192.168.141.31 /bin/controller[1]: {\"level\":\"debug\",\"msg\":\"Writing message to listener\",\"time\":\"2017-12-21T05:38:57Z\"}", :exception=>#<LogStash::Json::ParserError: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])"<7>2017-12-21T05:38:57Z ucp-controller-192.168.141.31 /bin/controller[1]: {"level":"debug","msg":"Writing message to listener","time":"2017-12-21T05:38:57Z"}"; line: 1, column: 2]>}

my logstash configuration is

input {
	tcp {
		port => 5000
		type => syslog
	}
}

## Add your filters / logstash plugins configuration here

output {
	elasticsearch {
		hosts => "elasticsearch:9200"
	}
}

filter {
    json {
        source => "message"
    }
}

As I understand from the logstash logs the UCP is sending this

<7>2017-12-21T05:38:57Z ucp-controller-192.168.141.31 /bin/controller[1]: {\"level\":\"debug\",\"msg\":\"Writing message to listener\",\"time\":\"2017-12-21T05:38:57Z\"}

in the message and this is not a valid json

What am I missing?

See https://docs.docker.com/datacenter/ucp/2.2/guides/admin/configure/store-logs-in-an-external-system/#example-setting-up-an-elk-stack
for reference


(Nicolas Bihan) #2

So, I was missing the right logstash configuration (documentation needs some update)

This configuration could be improved as there is some duplicate in the logs with it but at least this is searchable now.

input {
    tcp {
        port => 5000
        type => syslog
    }
}

## Add your filters / logstash plugins configuration here

output {
    elasticsearch {
        hosts => "elasticsearch:9200"
    }
}

filter {

    grok {
        match => { "message" => "%{SYSLOG5424PRI}%{TIMESTAMP_ISO8601:time} %{SYSLOGHOST} %{SYSLOGPROG}: %{GREEDYDATA:ucplog}"}
    }
    json {
        source => "ucplog"
    }
}

Result in Kibana