Heavy Forwarder REGEX Filting Issues

[dscoland]

Hi All,

I am having difficulty finding in-depth documentation on REGEX syntax, and I am attempting to filter out [WinEventLog:Security] logs from our central Splunk Instance by use of a heavy forwarder.

On our Heavy Forwarder, we have the following filters setup:

props.conf

[WinEventLog:Security]
TRANSFORMS-security= npm-setnull, everythingElse

Filter NPM account Security logs to nullQueue

[npm-setnull]
REGEX = (?msi)Account_Name="ACCOUNT@DOMAIN.COM"
DEST_KEY = queue
FORMAT = nullQueue

Send everything else to be indexed in the indexQueue

[everythingElse]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue

This is currently not filtering any of the logs, and I feel thate the syntax is incorrect, but I can't find any solid documentation on how to format this. Can anyone help?

Thank you in advance,r
Daniel

[kristian_kolb · Accepted solution]

You'll need to reverse the order in which the transforms are called from props.conf. Or in your case, just remove the [everythingElse] stanza and the reference to it. While it might seem a bit contra-intuitive, think of it like this;

"All events will pass through all transforms in the specified order before being returned for further processing"

It is not like there is some kind of 'break-on-match' processing. Thus, first setting the queue to nullQueue for some events, and then immediately resetting it back to indexQueue (for all events), will have the same results as not having any filtering at all.

---

UPDATE:

Yes, wolverine is right. The field name Account_Name is a so-called 'cleaned' name, i.e, spaces are replaced with underscores etc. The REGEX must match the actual text in the event, i.e.;

REGEX = Account\sName:\s+ACCOUNT@DOMAIN.COM

/k

/K

[kristian_kolb]

Yes, you need to restart splunkd - either from the outside, like restarting the service, or from the inside, in the GUI, or hit the refresh endpoint url;

http(s)://your_splunk:8000/en-US/debug/refresh

That will also reload most configs.

Purely search-related configs, such as field extractions will be reloaded for each search.

/k

[dscoland]

Thanks guys, I managed to get the filter working.

I changed the field to look for the Logon Account

[npm-setnull]
REGEX = Logon\sAccount:\snpm
DEST_KEY = queue
FORMAT = nullQueue

But I believe the Splunkd service in Services.msc needs to be rebooted in order to start the filter.

Regards!
Daniel

[dscoland]

Awesome, thanks Kristian!

As it turns out, this did not fix the issue. I will have to dig deeper in this.

Thanks for your continued help, guys.
Daniel

[kristian_kolb]

Hi dscoland,

www.regular-expressions.info

regexr.com

are two commonly referenced guides for learning/testing regular expressions.

[dscoland]

Hi wolverine!

Thank you for your response. I am going to try this out. I know I probably sound like a broken record. But does Splunk provide REGEX documentation? I feel rather ignorant not understanding (?msi) and (?m) syntax; or is this provided elsewhere?

Thanks for your help, guys.

Daniel

[kristian_kolb]

good spotting, wolverine! Updated my answer to reflect on this as well.

/k

[the_wolverine]

I think the issue might be with your REGEX:

REGEX = (?m)Account_Name="ACCOUNT@DOMAIN.COM"

[dscoland]

Hi Kristian,

Thank you for your response.

I now have:

[WinEventLog:Security]
TRANSFORMS-security= npm-setnull

[npm-setnull]
REGEX = (?msi)Account_Name="NPM@EDM.LOCAL"
DEST_KEY = queue
FORMAT = nullQueue

It still appears not to be filtering. Do I need to restart the Heavy Forwarder instance?

Daniel