Splunk logging driver logs not parsed by indexer

[ps]

Hi Splunkers,

I have start using Splunk Logging Driver to get my docker logs into Splunk. I am using Splunk Enterprice 8.0.1.

Problem is that indexer does not parse docker logs. I have tried with json and raw formats but either seems not to be noticed by indexer.

Current setup. HEC token used has source type _raw and all indexes allowed.

Docker startup

docker run \
> --log-driver=splunk \
> --log-opt splunk-token=xxxx \
> --log-opt splunk-url=http://xxxxx:8088 \
> --log-opt splunk-format=raw \
> --log-opt tag="{{.Name}}/{{.FullID}}" \
> --log-opt labels=location \
> --log-opt env=TEST \
> --env "TEST=false" \
> --label location=xxxxx \
> containerId

props.conf

[source=http:docker]
INDEXED_EXTRACTIONS=JSON
KV_MODE = none
AUTO_KV_JSON= false
TRANSFORMS-class_to_xx_index = route_to_xx_index

transforms.conf

[route_to_xx_index]
REGEX = .*\"xx\":\"xx\".*
DEST_KEY = _MetaData:Index
FORMAT = xx_index

All logs are going to default index. I have double checked that regex pattern matches and same pattern is working for universal forwarder, which logs are parsed and indexed correctly.

Input I get to default index is one line

containerName/container location=xx TEST=false {"message":"User xxx does xxx","priority":6,"priorityName":"INFO","sessionId":"xxx","action":"auth/login","application":"xx","environment":"development","security_level":"xx","info":"xxx"}

which does not get parsed and index.

 If I try with with _json token input to Splunk is "line" format and with same content and logs are also not parsed.

Any idea what I am doing wrong here. How to get json formatted logs to be parsed?

[juhatamminen · Accepted solution]

Hello,

when catching up source at props.conf stanza you have to use two colons instead of equal sign. Like

[source::http:docker]

For regex I would use capturing group, for example

REGEX = security_level\":\"([^"]*)
DEST_KEY = _MetaData:Index
FORMAT = $1

When event goes to xx -index. Or as hard coded 

REGEX = security_level\":\"xx\"
DEST_KEY = _MetaData:Index
FORMAT = xx_index

Event ends up to xx_index -index.

[ps]

up