topic Re: Logging multiple sources from Docker with HEC: Are multiple sourcetypes possible? in Getting Data In

Logging multiple sources from Docker with HEC: Are multiple sourcetypes possible?

twinspop — Mon, 26 Jun 2017 20:51:20 GMT

We have a group using HEC to deliver logs from Docker, but there are many different types of logs in the stream. It appears that any logs they want included in the HEC stream are written to STDOUT, but that means they all show as one sourcetype. Am I missing something? I've got app logs, error logs with stack traces, and web access logs. Each has their own log style as well as time format.

Am I stuck with overriding sourcetypes with a transform, or is there a better way?

Thanks!

Re: Logging multiple sources from Docker with HEC: Are multiple sourcetypes possible?

cpetterborg — Mon, 26 Jun 2017 23:13:54 GMT

Do they have control over how the data is getting to the HEC? If so, then they can set index, sourcetype, etc. in the HTTP request to the HEC. That would likely be the best way.

If they don't have control, then it would likely be best to use transforms in the indexers.

Re: Logging multiple sources from Docker with HEC: Are multiple sourcetypes possible?

lguinn2 — Tue, 27 Jun 2017 00:20:19 GMT

If each type of data is sent to the stream with a different token, then the HEC can easily separate out the different types of data.
This is a good reference: Set up and use HTTP Event Collector
http://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector

The section "Create an Event Collector token" shows how to create multiple tokens.

Re: Logging multiple sources from Docker with HEC: Are multiple sourcetypes possible?

twinspop — Tue, 27 Jun 2017 01:20:15 GMT

This would be grand if you can start up docker with multiple tokens. AFAIK, you cannot. Did I miss one of the Splunk driver options?

Re: Logging multiple sources from Docker with HEC: Are multiple sourcetypes possible?

twinspop — Tue, 27 Jun 2017 01:21:21 GMT

They control the docker instance, if that's what you mean. But I don't see a way to classify different sources coming from 1 docker container as different sourcetypes.

Re: Logging multiple sources from Docker with HEC: Are multiple sourcetypes possible?

lguinn2 — Wed, 28 Jun 2017 11:45:26 GMT

If you can't use multiple tokens, then you can still parse the data stream into multiple sourcetypes. You will need to use transforms on the indexers (or heavy forwarders) where the HEC runs.