How to configure props.conf and transforms.conf to filter out a Windows eventcode?

[pdjhh]

Hi guys,

I am ingesting Windows event logs including event code 5156 which is chewing up a lot of license. I have had it turned off (as it is 'firewall permitted connection'), but we want to track connections from these hosts to external IP addresses so I have switched it back on. What I'm thinking is, it would be good to be able to filter out all of the events that are destined for the internal network as they are not of interest.

The internal network is 192.168.x.x so I'd be looking to filter out all of those and let the rest get indexed. That would mean filtering out EventCode=4624 with field Destination Address=192.168*.

Looking around this will probably be a props and transforms with regex or potentially a fancy blacklist. Hope you can advise.

Thanks.

[Runals · Accepted solution]

You could also look at re-writing the events to make them smaller = less license: http://runals.blogspot.com/2014/07/taming-verbose-windows-logs-in-splunk.html

[pdjhh]

Hi there. This did indeed work for me and I got my events down from 766 to about 170. Looking forward to seeing the license usage tomorrow. I had to do some new field extractions but the trimmed events were neat and tidy so easy to do. Thanks a lot.

[jeffland]

I'd say you need a black- and then a whitelist, so something like this in transforms.conf:

[throwout]
REGEX = (?m)^EventCode=(5156)
DEST_KEY = queue
FORMAT = nullQueue

[keepsome]
REGEX = Destination\sAddress\:\s192\.168\.
DEST_KEY = queue
FORMAT = indexQueue

and this in props.conf:

[WinEventLog:Security]
TRANSFORMS-set= throwout,keepsome

---

PS: I just realized that the above does the opposite of what you intended to do - it throws out all allowed firewall events, and it keeps specifically those destined for your local network. I must've misread your question halfway through, sorry.
One possibility to achieve your goal is this, where we assume there are no other occurences of "Destination Adress = some ip" in any of the windows events (which I'm not really sure about, but preliminary googling revealed these to be most common in events 5156 and 5152). Props.conf:

[WinEventLog:Security]
TRANSFORMS-set= throwoutlocal

and transforms.conf:

[throwoutlocal]
REGEX = Destination\sAddress\:\s192\.168\.
DEST_KEY = queue
FORMAT = nullQueue

A more reliable solution would be to use the cool method runals provided in the other answer - you won't miss any events at all, and you'll still cut down on the license and storage capacity for those prattling windows event logs.

[pdjhh]

Thanks for getting back to me. I'm very new to regex but that looks like you're keeping the events with 192.168. in them? I need to drop those ones and keep the rest. What do you think?

[jeffland]

I have already added something to the answer in the meantime, does that help you out?

[pdjhh]

Mmm, I'm still getting the events with destination_address=192.168.x.x

[jeffland]

Check your raw windows event logs. I based the above regex on a sample event I found for that event code, maybe yours looks differently? Please post a raw event.

[pdjhh]

08/11/2015 05:48:23 PM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=5156
EventType=0
Type=Information
ComputerName=vm50_Template
TaskCategory=Filtering Platform Connection
OpCode=Info
RecordNumber=29313482
Keywords=Audit Success
Message=The Windows Filtering Platform has permitted a connection.

Application Information:
Process ID: 1628
Application Name: \device\harddiskvolume2\program files\vmware\vmware view\agent\bin\wsnm_jms.exe

Network Information:
Direction: Outbound
Source Address: 192.168.10.200
Source Port: 49159
Destination Address: 192.168.6.21
Destination Port: 4001
Protocol: 6

Filter Information:
Filter Run-Time ID: 76977
Layer Name: Connect
Layer Run-Time ID: 48

[jeffland]

The regex should work then. Strange.
You did restart the forwarder I assume? Can you confirm the settings with btool?

[pdjhh]

I had the .conf files in the wrong location on the forwarder! Moved them to etc/system/local and started to take effect. Based on that I moved onto the solution mentioned by runals below which also worked after some addtional field extractions I had to do on the newly trimmed events.

Thanks guys.

[pdjhh]

IT could be working now I'm not sure as I'm not getting any hits. The connections to non 192 addresses are few and far between so I'll leave it run for a while. If I see any events destined for non 192 addresses then we must be there..

[pdjhh]

I'm seeing a huge number less events but still ones with 192 destination addresses. Problem with that is I don't really know what's working and what isn't. Will need to look deeper. Thanks.

[pdjhh]

I've just thrown it in, will need to let it run for a while so thanks, will write back.