topic Re: Props and transforms to strip syslog header from Zeek data in Getting Data In

Props and transforms to strip syslog header from Zeek data

robnewman666 — Wed, 07 Apr 2021 09:26:38 GMT

I am trying to strip the Syslog header from the Zeek data that I have coming in as the Corelight TA only likes the raw zeek files.

At the moment I have (on a clustered network) -on the indexers in /opt/splunk/etc/system/local the following transforms.conf and below that the props.conf:

transforms.conf:

[syslog-header-stripper-ts-host]

REGEX = ^<\d+>[A-Z][a-z]+\s+\d+\s+\d+:\d+:\d+\s[^\s]*\s\S+:\s(.*)$

FORMAT = $1

DEST_KEY = _raw

props.conf:

[syslog]

# For zeek data - stripping the syslog header

TRANSFORMS-strip-syslog = syslog-header-stripper-ts-host

This doesn't seem to work for the data - as it is still arriving at the Search Heads with the Syslog header on it. Do I need to put these onto the Search Heads instead? Or does the props and transforms need editing?

Re: Props and transforms to strip syslog header from Zeek data

richgalloway — Wed, 07 Apr 2021 12:13:09 GMT

It's very difficult to debug a regular expression without sample data. Please provide some.

Consider using the SEDCMD setting in props.conf. It needs no transforms.

SEDCMD-noheader = s/^<\d+>[A-Z][a-z]+\s+\d+\s+\d+:\d+:\d+\s[^\s]*\s\S+:\s//

Re: Props and transforms to strip syslog header from Zeek data

robnewman666 — Wed, 07 Apr 2021 12:17:52 GMT

Thanks, will give this a try

Re: Props and transforms to strip syslog header from Zeek data

robnewman666 — Wed, 07 Apr 2021 12:57:05 GMT

This did work a treat. Thanks very much!