topic Re: Transforms and Props for Linux Syslog Filtering in Getting Data In

Transforms and Props for Linux Syslog Filtering

gkstev87 — Sat, 16 Mar 2024 08:08:25 GMT

Hello,

Our customer has decided to end use of Splunk in lieu of Sumo Logic, but we are looking to keep up internal use of Splunk due to 110GB worth of Perpetual licensing we have leftover. We are currently filtering out non-essentials, and for us one of the big players is linux syslog. I am attempting to use transforms and props to filter out everything that aren't authentication failures. The regular expression is looking for the string of text "authentication failure". I tested my regex in regex101 and everything checks out, but when I turn on the syslog sourcetype, the proverbial flood gates are still opening up.

Can someone take a look at these and let me know what looks wrong here? The transforms are meant to bring in only events with "authentication failure" and toss out everything else.

Props.conf [syslog] TRANSFORMS-set=set_parsing,set_null Transforms.conf [set_parse] REGEX = \bauthentication\b\s\bfailure\b DEST_KEY = queue FORMAT = indexQueue [set_null] REGEX = . DEST_KEY = queue FORMAT = nullQueue

Re: Transforms and Props for Linux Syslog Filtering

richgalloway — Thu, 12 Oct 2023 14:34:13 GMT

The regex is looking for 3 characters between "authentication" and "failure", but the event contains only 1. Try

REGEX = \bauthentication\sfailure\b

Re: Transforms and Props for Linux Syslog Filtering

gkstev87 — Thu, 12 Oct 2023 18:50:49 GMT

Thank you for your help here Rich! I replaced the regex with the one you provided and still am not seeing any data at all. Both my regular expression, as well as the one you provided, worked to locate the string "authentication failure" in regex101. This has left me more or less stumped. It's like the Heavy Forwarder is completely ignoring the "set_parse" setting. Perhaps the issue lies with props.conf? I have use props and transforms before, just not sure what the issue is here.

What do you think?

Thanks

Grant

Re: Transforms and Props for Linux Syslog Filtering

richgalloway — Fri, 13 Oct 2023 00:30:28 GMT

It's possible Splunk's regex library doesn't handle \b well. Is there something else that indicates the start and end of the desired string? Perhaps \Wauthentication\sfailure\W?

Re: Transforms and Props for Linux Syslog Filtering

Sucheta_new — Sat, 16 Mar 2024 08:07:49 GMT

Hi,

Can you try the below config in props:

[syslog] TRANSFORMS-set=set_parse,set_null

As your transforms stanza says set_parse and props set_parsing.

Re: Transforms and Props for Linux Syslog Filtering

jotne — Sun, 17 Mar 2024 05:47:12 GMT

Why not just:

REGEX = authentication\sfailure