topic Event Filtering not working in Getting Data In

Event Filtering not working

jambajuice — Fri, 25 Jun 2010 00:52:11 GMT

We are trying to filter events from the Windows Event Log that are pulled using WMI.

Here is the transforms.conf:

[WMInull]
REGEX = (?m)^EventCode=(538|562|571|573|576|577|578|672|680|4634|4658|4672|4673|4674|4690|4772|4776|5156)
DEST_KEY = queue
FORMAT = nullQueue

Here is the props.conf:

[WMI:WinEventLog:Security]
EXTRACT-db_user = (?i) user '(?P<db_user>[^']*)'
TRANSFORMS-null = WMInull

But if I do a search of the security log and use | regex _raw="(?m)^EventCode=(538|562|571|573|576|577|578|672|680|4634|4658|4672|4673|4674|4690|4772|4776|5156)", I am seeing events that should have been filtered.

Any idea what the problem is? It was working prior to the upgrade to 4.x

Re: Event Filtering not working

Lowell — Fri, 25 Jun 2010 01:57:18 GMT

Is your sourcetype correct? I'm not familiar with collecting event logs via WMI (which if I understand correctly, is not recommended.) So if your sourcetype is wrong, then that would certainly cause a problem.

Update: Based on gkanapathy note. You could simply use the sourcetype of wmi. That's a lot of overhead if your collecting a fair about of WMI events though. If you use the WinEventLog inputs directly (instead of pulling them via WMI), then you could use the sourcetype of [WinEventLog:Security] (since the windows event logs are not assigned a sourcetype dynamically like the wmi events are) so this would be more efficient because your transformer would run for fewer events.

If you have to collect these using WMI, you may also want to consider also making your regex match wmi_type=WinEventLog:Security which would prevent your regex from matching against other Application or System logs, but then again that will make the regex-overhead even higher....

Side note: One thing I would certainly suggested is that you update your REGEX to ensure that whatever comes after your code isn't another digit. For example, you want to block 538 but not necessarily 5381 or 5389999999 (I'm not that familiar with all the different codes here, but it certainly seems like you could be matching more than you want.)

REGEX = (?m)^EventCode=(538|562|571|573|576|577|578|672|680|4634|4658|4672|4673|4674|4690|4772|4776|5156)\b

Re: Event Filtering not working

gkanapathy — Fri, 25 Jun 2010 02:43:33 GMT

I am surprised that it used to work, because a props.conf rule on [WMI:WinEventLog:Security] does not get invoked at index time (so TRANSFORMS) will not work. This is because the sourcetype of data collected via WMI is not set until index time, so when rules are selected that one will not work. (It does work for search time rules, e.g., REPORT.) The sourcetype at that point is just wmi, and this applies to all WMI-collected data.

Re: Event Filtering not working

jambajuice — Fri, 25 Jun 2010 02:53:33 GMT

Then what is the best practice for filtering events collected with WMI?

Re: Event Filtering not working

the_wolverine — Fri, 25 Jun 2010 03:21:04 GMT

You most certainly can filter wmi events. The method, which is to reference the wmi sourcetype, is discussed in this post:

http://answers.splunk.com/questions/3239/try-to-route-certain-wmi-events-to-nullqueue