Splunk Search

Filtering events out via props.conf and transforms.conf?

ejdavis
Path Finder

The props.conf and transforms.conf files that should be modified are under /etc/system/local, correct?

We have been unable to get filtering to work at all, and just to test it we are attempting to filter out all Event Code 560's coming from source WMI:WinEventLog:Security

In props.conf we have attempted:

[WMI:WinEventLog:Security]
TRANSFORMS-FilterEvent560 = FilterEvent560

We have also tried

[WinEventLog:Security]
[source::WMI:WinEventLog:Security]
[source::WinEventLog:Security]

Then in transforms.conf we have attempted:

[FilterEvent560]
REGEX = (?msi)^EventCode=560
DEST_KEY = queue
FORMAT = nullQueue

We have also tried

REGEX = (?m)^EventCode=560
REGEX = (?ms)^EventCode=560
REGEX = (?m)^EventCode=\560
REGEX = (?msi)^EventCode=(560)

And a lot of other variations. All forms of the regex work via | regex _raw in the search function, but we have not been able to filter out events. Is there something that we're missing?

0 Karma
1 Solution

lguinn2
Legend

Where are you putting the props.conf and the transforms.conf? These settings need to be on the system where the data is parsed.

If you are using a Universal Forwarder to collect this data, the parsing will occur on the indexer(s).

If you are using a Heavy Forwarder to collect this data, the parsing will occur on the forwarder.

Also, if you are using Splunk 6.0, you can filter on the event codes in a new way. Read more about that in this blog entry on Windows Event Logs in Splunk 6.0

View solution in original post

lukejadamec
Super Champion

I don't use WMI inputs, but this should work. I'd put it in the app/windows/local folder. Also, you might try making the TRANSFORMS-label different than the = identifier.

Props.conf

[source::WMI:WinEventLog:Security]
TRANSFORMS-FilterEvent = FilterEvent560

Transforms.conf

[FilterEvent560]
REGEX = (?msi)^EventCode=560
DEST_KEY = queue
FORMAT = nullQueue

ejdavis
Path Finder

Ah, okay. I don't know what fixed it but I deleted recreated the props.conf and transforms.conf on the heavy forwarder and it started filtering the events out.

Thanks much, I've got a much better understanding of the system now.

0 Karma

lukejadamec
Super Champion

I'm not saying you have to, just that I do because it helps to keep things together.
Are you sure it is a WMI input?
If the source specified in props.conf is not correct, then it is not going to work.
Create a search for the data you are trying to filter, and verify the Source and Sourcetype.

ejdavis
Path Finder

I have restarted them both after the changes.

I could not find a WMI.conf anywhere on the indexer or heavy forwarder. This is the first time that I'm seeing that you need to put the changes anywhere other than props.conf or transforms.conf in /etc/system/local

0 Karma

lukejadamec
Super Champion

Did you restart them both (index time transforms require a Splunk restart)?

Did find the location of the WMI.conf that configures the inputs?

ejdavis
Path Finder

I currently have an indexer at my location and a heavy forwarder installed a store location. At the heavy forwarder, it is pulling from multiple terminals and POS devices via WMI. I do not have a Windows app installed. I put the props.conf and transforms.conf files in /etc/system/local on the heavy forwarder and indexer, and the events are still being received at the indexer.

0 Karma

lukejadamec
Super Champion

I changed my mind about Iguinn's comment. WMI data is typically pulled from the indexer, so that is where the configs should be. Unless you're pulling the WMI data locally, which does not make much sense.
Regardless, what I do is I put the props and transforms configs in the local directory for the app that configures the input.
For the WMI filter test I just ran, I put the configs in the Windows app local folder on the indexer.
You should put them where the input is configured on the forwarder (if the WMI input is configured on the forwarder), or do like I did and put them on the indexer.

ejdavis
Path Finder

Thank you.

Now, this may be a stupid question, but where should that props.conf and transforms.conf file be? In /etc/system/local? I'm not sure where I'd find app/windows/local.

0 Karma

lukejadamec
Super Champion

I just tested a variant of this for WMI:CPUTime, PercentProcessorTime=0, and it works fine. Like Iguinn said, it is probably because your configs are not on the heavy forwarder.... The data can only be cooked once.

lguinn2
Legend

Where are you putting the props.conf and the transforms.conf? These settings need to be on the system where the data is parsed.

If you are using a Universal Forwarder to collect this data, the parsing will occur on the indexer(s).

If you are using a Heavy Forwarder to collect this data, the parsing will occur on the forwarder.

Also, if you are using Splunk 6.0, you can filter on the event codes in a new way. Read more about that in this blog entry on Windows Event Logs in Splunk 6.0

lguinn2
Legend

etc/system/local is a fine place for the props.conf and transforms.conf

Did you restart the heavy forwarder after making these changes? Also note that this will affect only new data; events that have already been forwarded will be unaffected.

0 Karma

ejdavis
Path Finder

I have added the code to props.conf and transforms.conf on the heavy forwarder, but events are still coming through.

Just to clarify, which props.conf and transforms.conf should I be editing? In /etc/system/local?

0 Karma

ejdavis
Path Finder

I am editing the props.conf and transforms.conf in /etc/system/local on the server that is actually doing the indexing.

So I will edit the files on my heavy forwarders and see if that works. I will also take a look into the Splunk 6.0 event code filtering. Thanks a lot.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...