Splunk Search

Filtering events out via props.conf and transforms.conf?

Path Finder

The props.conf and transforms.conf files that should be modified are under /etc/system/local, correct?

We have been unable to get filtering to work at all, and just to test it we are attempting to filter out all Event Code 560's coming from source WMI:WinEventLog:Security

In props.conf we have attempted:

[WMI:WinEventLog:Security]
TRANSFORMS-FilterEvent560 = FilterEvent560

We have also tried

[WinEventLog:Security]
[source::WMI:WinEventLog:Security]
[source::WinEventLog:Security]

Then in transforms.conf we have attempted:

[FilterEvent560]
REGEX = (?msi)^EventCode=560
DEST_KEY = queue
FORMAT = nullQueue

We have also tried

REGEX = (?m)^EventCode=560
REGEX = (?ms)^EventCode=560
REGEX = (?m)^EventCode=\560
REGEX = (?msi)^EventCode=(560)

And a lot of other variations. All forms of the regex work via | regex _raw in the search function, but we have not been able to filter out events. Is there something that we're missing?

0 Karma
1 Solution

Legend

Where are you putting the props.conf and the transforms.conf? These settings need to be on the system where the data is parsed.

If you are using a Universal Forwarder to collect this data, the parsing will occur on the indexer(s).

If you are using a Heavy Forwarder to collect this data, the parsing will occur on the forwarder.

Also, if you are using Splunk 6.0, you can filter on the event codes in a new way. Read more about that in this blog entry on Windows Event Logs in Splunk 6.0

View solution in original post

Super Champion

I don't use WMI inputs, but this should work. I'd put it in the app/windows/local folder. Also, you might try making the TRANSFORMS-label different than the = identifier.

Props.conf

[source::WMI:WinEventLog:Security]
TRANSFORMS-FilterEvent = FilterEvent560

Transforms.conf

[FilterEvent560]
REGEX = (?msi)^EventCode=560
DEST_KEY = queue
FORMAT = nullQueue

Path Finder

Ah, okay. I don't know what fixed it but I deleted recreated the props.conf and transforms.conf on the heavy forwarder and it started filtering the events out.

Thanks much, I've got a much better understanding of the system now.

0 Karma

Super Champion

I'm not saying you have to, just that I do because it helps to keep things together.
Are you sure it is a WMI input?
If the source specified in props.conf is not correct, then it is not going to work.
Create a search for the data you are trying to filter, and verify the Source and Sourcetype.

Path Finder

I have restarted them both after the changes.

I could not find a WMI.conf anywhere on the indexer or heavy forwarder. This is the first time that I'm seeing that you need to put the changes anywhere other than props.conf or transforms.conf in /etc/system/local

0 Karma

Super Champion

Did you restart them both (index time transforms require a Splunk restart)?

Did find the location of the WMI.conf that configures the inputs?

Path Finder

I currently have an indexer at my location and a heavy forwarder installed a store location. At the heavy forwarder, it is pulling from multiple terminals and POS devices via WMI. I do not have a Windows app installed. I put the props.conf and transforms.conf files in /etc/system/local on the heavy forwarder and indexer, and the events are still being received at the indexer.

0 Karma

Super Champion

I changed my mind about Iguinn's comment. WMI data is typically pulled from the indexer, so that is where the configs should be. Unless you're pulling the WMI data locally, which does not make much sense.
Regardless, what I do is I put the props and transforms configs in the local directory for the app that configures the input.
For the WMI filter test I just ran, I put the configs in the Windows app local folder on the indexer.
You should put them where the input is configured on the forwarder (if the WMI input is configured on the forwarder), or do like I did and put them on the indexer.

Path Finder

Thank you.

Now, this may be a stupid question, but where should that props.conf and transforms.conf file be? In /etc/system/local? I'm not sure where I'd find app/windows/local.

0 Karma

Super Champion

I just tested a variant of this for WMI:CPUTime, PercentProcessorTime=0, and it works fine. Like Iguinn said, it is probably because your configs are not on the heavy forwarder.... The data can only be cooked once.

Legend

Where are you putting the props.conf and the transforms.conf? These settings need to be on the system where the data is parsed.

If you are using a Universal Forwarder to collect this data, the parsing will occur on the indexer(s).

If you are using a Heavy Forwarder to collect this data, the parsing will occur on the forwarder.

Also, if you are using Splunk 6.0, you can filter on the event codes in a new way. Read more about that in this blog entry on Windows Event Logs in Splunk 6.0

View solution in original post

Legend

etc/system/local is a fine place for the props.conf and transforms.conf

Did you restart the heavy forwarder after making these changes? Also note that this will affect only new data; events that have already been forwarded will be unaffected.

0 Karma

Path Finder

I have added the code to props.conf and transforms.conf on the heavy forwarder, but events are still coming through.

Just to clarify, which props.conf and transforms.conf should I be editing? In /etc/system/local?

0 Karma

Path Finder

I am editing the props.conf and transforms.conf in /etc/system/local on the server that is actually doing the indexing.

So I will edit the files on my heavy forwarders and see if that works. I will also take a look into the Splunk 6.0 event code filtering. Thanks a lot.

0 Karma