Getting Data In

Props and transforms to strip syslog header from Zeek data

robnewman666
Path Finder

I am trying to strip the Syslog header from the Zeek data that I have coming in as the Corelight TA only likes the raw zeek files.

At the moment I have (on a clustered network) -on the indexers in /opt/splunk/etc/system/local the following transforms.conf and below that the props.conf:

 

transforms.conf:

 

[syslog-header-stripper-ts-host]

REGEX = ^<\d+>[A-Z][a-z]+\s+\d+\s+\d+:\d+:\d+\s[^\s]*\s\S+:\s(.*)$

FORMAT = $1

DEST_KEY = _raw

 

props.conf:

[syslog]

# For zeek data - stripping the syslog header

TRANSFORMS-strip-syslog = syslog-header-stripper-ts-host

This doesn't seem to work for the data - as it is still arriving at the Search Heads with the Syslog header on it. Do I need to put these onto the Search Heads instead? Or does the props and transforms need editing?

Labels (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

It's very difficult to debug a regular expression without sample data.  Please provide some.

Consider using the SEDCMD setting in props.conf.  It needs no transforms.

SEDCMD-noheader = s/^<\d+>[A-Z][a-z]+\s+\d+\s+\d+:\d+:\d+\s[^\s]*\s\S+:\s//
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

robnewman666
Path Finder

This did work a treat. Thanks very much!

0 Karma

richgalloway
SplunkTrust
SplunkTrust

It's very difficult to debug a regular expression without sample data.  Please provide some.

Consider using the SEDCMD setting in props.conf.  It needs no transforms.

SEDCMD-noheader = s/^<\d+>[A-Z][a-z]+\s+\d+\s+\d+:\d+:\d+\s[^\s]*\s\S+:\s//
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

robnewman666
Path Finder

Thanks, will give this a try

 

0 Karma