topic Remove syslog prefix from Json in Getting Data In

Remove syslog prefix from Json

vbotnari1 — Thu, 11 Jul 2019 15:23:44 GMT

I have a Json log which looks like this

Jul 11 14:37:48 darktrace-dt-722-01 darktrace {"creationTime":1562855937000,"breachUrl":...}

I have to remove the timestamp hostanem, all syslog prefixes until {

This is how my props.conf looks like

[darktrace]
SEDCMD-StripHeader = ^([^\{]+) 
KV_MODE = json
pulldown_type = true
category = Structured
description = darktrace

But it doesn't work. I tried INDEXED_EXTRACTIONS = json as well without success.

Any help is appreciated. Thanks

Re: Remove syslog prefix from Json

woodcock — Thu, 11 Jul 2019 18:55:18 GMT

It must be an actual sed command like this:

 SEDCMD-StripHeader = s/^[^\{]+//

Re: Remove syslog prefix from Json

vbotnari1 — Fri, 12 Jul 2019 07:29:03 GMT

Thank you @woodcock . I tried your suggested sed command but it did nothing.

Re: Remove syslog prefix from Json

woodcock — Fri, 12 Jul 2019 13:58:48 GMT

If you are doing a sourcetype override/overwrite, you must use the ORIGINAL value NOT the new value, then you must deploy this to the first full instance(s) of Splunk that handles the events (usually either the HF-tier, if you use this, or your Indexer tier), restart all Splunk instances there, send in new events (old events will stay broken), then test using _index_earliest=-5m to be absolutely certain that you are only examining the newly indexed events.