topic Re: WinEventLog filtering on indexer in Getting Data In

WinEventLog filtering on indexer

CerielTjuh — Fri, 16 Apr 2010 19:35:34 GMT

Hello there,

I have currently deployed Splunk in our network using SplunkLightForwarders and one central indexing server. I am indexing Windows Event Logs and forwarding them to the central indexer.

I am trying to create a filter to filter and send only the eventcode wich we want to see. I know a LightForwarder doesn't have the ability to filter the data so the action needs to be done on the central indexer.

I have created a props.conf and transforms.conf but it doesn't seem to work and i am confused why it doens't work.

props.conf

[WinEventLog:Security]
TRANSFORMS-queue=InputAllowed,InputNull

transforms.conf

[InputAllowed]
REGEX=^EventCode=(4634|4662)
DEST_KEY=queue
FORMAT=indexQueue

[InputNull]
REGEX=(.)
DEST_KEY=queue
FORMAT=nullQueue

If I replace the configuration so that the EventCodes 4634 and 4662 are droppen to the nullQueue it works, only the filtering so that everything else is dropped doens't work . . .

Re: WinEventLog filtering on indexer

BunnyHop — Fri, 16 Apr 2010 19:41:24 GMT

Please see this:

http://answers.splunk.com/questions/577/how-do-you-filter-windows-event-log

This talks about filtering the native Windows Event Log. Since the eventlog is a multiline type event log, you will have to add the (?m) before your query. Also, if you're using LWF, you would want to have the props.conf and transforms.conf configured on the indexer. I would do a negate regex, so that way everything is not indexed but eventCodes 4634 and 4662. By doing this, you can remove the InputNull reference on your props, and completely remove it from your transform. Change your transform to this:

transforms.conf

[WinEventLog:Security]
TRANSFORMS-InputNegate = InputNegate

transforms.conf

[InputNegate]
REGEX = (?msi).*EventCode=([^4634]|[^4662]).*
DEST_KEY = queue
FORMAT = nullQueue

Re: WinEventLog filtering on indexer

CerielTjuh — Fri, 16 Apr 2010 21:19:19 GMT

True, changed it but still not receiving anything.
The funny fact is if i change the InputNull to remove the eventcodes 4634 and leave the other one unchanged it works (so he removes the events with eventcode 4634 and forwards the others) but i want to be able to keep the events that i want to see and remove all the others

Re: WinEventLog filtering on indexer

BunnyHop — Sat, 17 Apr 2010 01:00:23 GMT

Then in that case, I've updated my original answer.

Re: WinEventLog filtering on indexer

CerielTjuh — Mon, 19 Apr 2010 13:25:39 GMT

I'm sorry but it still isn't working Bunny, EventCode 4769 is also in my results... Not sure if I gave you a good idea of what i want to do, i want to create a filter of the events i want to see, not remove the things i don't want to see. During the weekend i tried multiple things on my environment and also on a fresh installed environment without results. Is it even possible ?

Re: WinEventLog filtering on indexer

BunnyHop — Mon, 19 Apr 2010 19:45:18 GMT

It should be possible. What are you trying to accomplish? If you want to keep the rest of the events on Splunk you're probably better off creating a saved search instead of filtering.

Re: WinEventLog filtering on indexer

CerielTjuh — Mon, 19 Apr 2010 20:08:24 GMT

The problem is that my license doesn't allow it 😉

Re: WinEventLog filtering on indexer

BunnyHop — Tue, 20 Apr 2010 03:17:39 GMT

Understood. The problem really is either you're blacklisting or whitelisting. You can either allow for certain events and then drop everything else or you can drop certain events and allow everything else. Does this make sense?

Re: WinEventLog filtering on indexer

CerielTjuh — Wed, 21 Apr 2010 13:28:11 GMT

Yes, and im trying to create a whitelist, that is what i want in the end, but the problem is that it doesn't work, if I change the FORMAT = nullQueue to indexQueue the events aren't showing up. Or do I need to make a different whitelist?

Re: WinEventLog filtering on indexer

CerielTjuh — Thu, 17 Jun 2010 01:53:00 GMT

Any new thoughts on this issue?

-props.conf-
[source::WinEventLog:Security]
TRANSFORMS-set= setparsing, setnull

Part two.

-transforms.conf-
[setparsing]
REGEX = (?m)^EventCode=(592|593)
DEST_KEY = queue
FORMAT = tcpOutQueue

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

The funny side is that Splunk does not forward anything when i activate this script. Only the Application and System logs are forwarding. I also tried to publish this application as a deployment app, but that doesn't work either...

I could really use some help 😉

Re: WinEventLog filtering on indexer

gkanapathy — Thu, 17 Jun 2010 02:16:45 GMT

you should reverse the order. if you set the queue to tcpOutQueue (which I don't believe is correct anyway), the next rule sets it to null, so it will be discarded. It's also important to specify whether this is on an indexer or light forwarder or regular forwarder.

Re: WinEventLog filtering on indexer

CerielTjuh — Fri, 18 Jun 2010 13:30:50 GMT

I will change the order and try again, I tried it on a light forwarder (but that doesn't work by design) and now on a forwarder (basic install with a deployment app to forward the data)

Re: WinEventLog filtering on indexer

Lowell — Sat, 28 Aug 2010 07:56:17 GMT

CerielTjuh, please update your question. If you have found a solution, please indicate so by checking one of them.

Re: WinEventLog filtering on indexer

gfriedmann — Tue, 18 Jan 2011 06:55:04 GMT

gkanapathy: according to documentation, nullQueue transforms are processed last, so the order shouldn't matter in this case.
http://www.splunk.com/base/Documentation/latest/Admin/Routeandfilterdata

Re: WinEventLog filtering on indexer

BP9906 — Mon, 05 Nov 2012 17:04:42 GMT

your props.conf looks wrong.

[source::WinEventLog:Security]
TRANSFORMS-nullQ=nullFilter-WFP5156

transform.conf for my props.conf entry:

[nullFilter-WFP5156]
REGEX = EventCode=5156
DEST_KEY = queue
FORMAT = nullQueue