Props and transforms to strip syslog header from Zeek data

[robnewman666]

I am trying to strip the Syslog header from the Zeek data that I have coming in as the Corelight TA only likes the raw zeek files.

At the moment I have (on a clustered network) -on the indexers in /opt/splunk/etc/system/local the following transforms.conf and below that the props.conf:

transforms.conf:

[syslog-header-stripper-ts-host]

REGEX = ^<\d+>[A-Z][a-z]+\s+\d+\s+\d+:\d+:\d+\s[^\s]*\s\S+:\s(.*)$

FORMAT = $1

DEST_KEY = _raw

props.conf:

[syslog]

# For zeek data - stripping the syslog header

TRANSFORMS-strip-syslog = syslog-header-stripper-ts-host

This doesn't seem to work for the data - as it is still arriving at the Search Heads with the Syslog header on it. Do I need to put these onto the Search Heads instead? Or does the props and transforms need editing?

[robnewman666]

This did work a treat. Thanks very much!

[richgalloway · Accepted solution]

It's very difficult to debug a regular expression without sample data.  Please provide some.

Consider using the SEDCMD setting in props.conf.  It needs no transforms.

SEDCMD-noheader = s/^<\d+>[A-Z][a-z]+\s+\d+\s+\d+:\d+:\d+\s[^\s]*\s\S+:\s//

[robnewman666]

Thanks, will give this a try