Splunk Search

Heavy Forwarder REGEX Filting Issues

Path Finder

Hi All,

I am having difficulty finding in-depth documentation on REGEX syntax, and I am attempting to filter out [WinEventLog:Security] logs from our central Splunk Instance by use of a heavy forwarder.

On our Heavy Forwarder, we have the following filters setup:

props.conf

[WinEventLog:Security]
TRANSFORMS-security= npm-setnull, everythingElse

Filter NPM account Security logs to nullQueue

[npm-setnull]
REGEX = (?msi)Account_Name="ACCOUNT@DOMAIN.COM"
DEST_KEY = queue
FORMAT = nullQueue

Send everything else to be indexed in the indexQueue

[everythingElse]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue

This is currently not filtering any of the logs, and I feel thate the syntax is incorrect, but I can't find any solid documentation on how to format this. Can anyone help?

Thank you in advance,r
Daniel

0 Karma
1 Solution

Ultra Champion

You'll need to reverse the order in which the transforms are called from props.conf. Or in your case, just remove the [everythingElse] stanza and the reference to it. While it might seem a bit contra-intuitive, think of it like this;

"All events will pass through all transforms in the specified order before being returned for further processing"

It is not like there is some kind of 'break-on-match' processing. Thus, first setting the queue to nullQueue for some events, and then immediately resetting it back to indexQueue (for all events), will have the same results as not having any filtering at all.


UPDATE:

Yes, wolverine is right. The field name Account_Name is a so-called 'cleaned' name, i.e, spaces are replaced with underscores etc. The REGEX must match the actual text in the event, i.e.;

REGEX = Account\sName:\s+ACCOUNT@DOMAIN.COM

/k

/K

View solution in original post

0 Karma

Ultra Champion

You'll need to reverse the order in which the transforms are called from props.conf. Or in your case, just remove the [everythingElse] stanza and the reference to it. While it might seem a bit contra-intuitive, think of it like this;

"All events will pass through all transforms in the specified order before being returned for further processing"

It is not like there is some kind of 'break-on-match' processing. Thus, first setting the queue to nullQueue for some events, and then immediately resetting it back to indexQueue (for all events), will have the same results as not having any filtering at all.


UPDATE:

Yes, wolverine is right. The field name Account_Name is a so-called 'cleaned' name, i.e, spaces are replaced with underscores etc. The REGEX must match the actual text in the event, i.e.;

REGEX = Account\sName:\s+ACCOUNT@DOMAIN.COM

/k

/K

View solution in original post

0 Karma

Ultra Champion

Yes, you need to restart splunkd - either from the outside, like restarting the service, or from the inside, in the GUI, or hit the refresh endpoint url;

http(s)://your_splunk:8000/en-US/debug/refresh

That will also reload most configs.

Purely search-related configs, such as field extractions will be reloaded for each search.

/k

0 Karma

Path Finder

Thanks guys, I managed to get the filter working.

I changed the field to look for the Logon Account

[npm-setnull]
REGEX = Logon\sAccount:\snpm
DEST_KEY = queue
FORMAT = nullQueue

But I believe the Splunkd service in Services.msc needs to be rebooted in order to start the filter.

Regards!
Daniel

0 Karma

Path Finder

Awesome, thanks Kristian!

As it turns out, this did not fix the issue. I will have to dig deeper in this.

Thanks for your continued help, guys.
Daniel

0 Karma

Ultra Champion

Hi dscoland,

www.regular-expressions.info

regexr.com

are two commonly referenced guides for learning/testing regular expressions.

0 Karma

Path Finder

Hi wolverine!

Thank you for your response. I am going to try this out. I know I probably sound like a broken record. But does Splunk provide REGEX documentation? I feel rather ignorant not understanding (?msi) and (?m) syntax; or is this provided elsewhere?

Thanks for your help, guys.

Daniel

0 Karma

Ultra Champion

good spotting, wolverine! Updated my answer to reflect on this as well.

/k

0 Karma

Champion

I think the issue might be with your REGEX:

REGEX = (?m)Account_Name="ACCOUNT@DOMAIN.COM"

Path Finder

Hi Kristian,

Thank you for your response.

I now have:

[WinEventLog:Security]
TRANSFORMS-security= npm-setnull

[npm-setnull]
REGEX = (?msi)AccountName="NPM@EDM.LOCAL"
DEST
KEY = queue
FORMAT = nullQueue

It still appears not to be filtering. Do I need to restart the Heavy Forwarder instance?

Daniel

0 Karma