topic Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode? in Splunk Search

How to configure props.conf and transforms.conf to filter out a Windows eventcode?

pdjhh — Mon, 10 Aug 2015 12:54:34 GMT

Hi guys,

I am ingesting Windows event logs including event code 5156 which is chewing up a lot of license. I have had it turned off (as it is 'firewall permitted connection'), but we want to track connections from these hosts to external IP addresses so I have switched it back on. What I'm thinking is, it would be good to be able to filter out all of the events that are destined for the internal network as they are not of interest.

The internal network is 192.168.x.x so I'd be looking to filter out all of those and let the rest get indexed. That would mean filtering out EventCode=4624 with field Destination Address=192.168*.

Looking around this will probably be a props and transforms with regex or potentially a fancy blacklist. Hope you can advise.

Thanks.

Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode?

jeffland — Mon, 10 Aug 2015 13:30:50 GMT

I'd say you need a black- and then a whitelist, so something like this in transforms.conf:

[throwout]
REGEX = (?m)^EventCode=(5156)
DEST_KEY = queue
FORMAT = nullQueue

[keepsome]
REGEX = Destination\sAddress\:\s192\.168\.
DEST_KEY = queue
FORMAT = indexQueue

and this in props.conf:

[WinEventLog:Security]
TRANSFORMS-set= throwout,keepsome

PS: I just realized that the above does the opposite of what you intended to do - it throws out all allowed firewall events, and it keeps specifically those destined for your local network. I must've misread your question halfway through, sorry.
One possibility to achieve your goal is this, where we assume there are no other occurences of "Destination Adress = some ip" in any of the windows events (which I'm not really sure about, but preliminary googling revealed these to be most common in events 5156 and 5152). Props.conf:

[WinEventLog:Security]
TRANSFORMS-set= throwoutlocal

and transforms.conf:

[throwoutlocal]
REGEX = Destination\sAddress\:\s192\.168\.
DEST_KEY = queue
FORMAT = nullQueue

A more reliable solution would be to use the cool method runals provided in the other answer - you won't miss any events at all, and you'll still cut down on the license and storage capacity for those prattling windows event logs.

Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode?

Runals — Mon, 10 Aug 2015 14:50:54 GMT

You could also look at re-writing the events to make them smaller = less license: http://runals.blogspot.com/2014/07/taming-verbose-windows-logs-in-splunk.html

Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode?

pdjhh — Tue, 11 Aug 2015 06:55:21 GMT

Thanks for getting back to me. I'm very new to regex but that looks like you're keeping the events with 192.168. in them? I need to drop those ones and keep the rest. What do you think?

Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode?

jeffland — Tue, 11 Aug 2015 07:09:27 GMT

I have already added something to the answer in the meantime, does that help you out?

Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode?

pdjhh — Tue, 11 Aug 2015 07:32:51 GMT

I've just thrown it in, will need to let it run for a while so thanks, will write back.

Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode?

pdjhh — Tue, 11 Aug 2015 07:40:10 GMT

Mmm, I'm still getting the events with destination_address=192.168.x.x

Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode?

jeffland — Tue, 11 Aug 2015 07:44:14 GMT

Check your raw windows event logs. I based the above regex on a sample event I found for that event code, maybe yours looks differently? Please post a raw event.

Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode?

pdjhh — Tue, 11 Aug 2015 07:52:43 GMT

08/11/2015 05:48:23 PM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=5156
EventType=0
Type=Information
ComputerName=vm50_Template
TaskCategory=Filtering Platform Connection
OpCode=Info
RecordNumber=29313482
Keywords=Audit Success
Message=The Windows Filtering Platform has permitted a connection.

Application Information:
Process ID: 1628
Application Name: \device\harddiskvolume2\program files\vmware\vmware view\agent\bin\wsnm_jms.exe

Network Information:
Direction: Outbound
Source Address: 192.168.10.200
Source Port: 49159
Destination Address: 192.168.6.21
Destination Port: 4001
Protocol: 6

Filter Information:
Filter Run-Time ID: 76977
Layer Name: Connect
Layer Run-Time ID: 48

Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode?

jeffland — Tue, 11 Aug 2015 07:58:07 GMT

The regex should work then. Strange.
You did restart the forwarder I assume? Can you confirm the settings with btool?

Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode?

pdjhh — Tue, 11 Aug 2015 08:39:34 GMT

IT could be working now I'm not sure as I'm not getting any hits. The connections to non 192 addresses are few and far between so I'll leave it run for a while. If I see any events destined for non 192 addresses then we must be there..

Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode?

pdjhh — Tue, 11 Aug 2015 08:58:41 GMT

I'm seeing a huge number less events but still ones with 192 destination addresses. Problem with that is I don't really know what's working and what isn't. Will need to look deeper. Thanks.

Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode?

pdjhh — Tue, 11 Aug 2015 12:41:27 GMT

I had the .conf files in the wrong location on the forwarder! Moved them to etc/system/local and started to take effect. Based on that I moved onto the solution mentioned by runals below which also worked after some addtional field extractions I had to do on the newly trimmed events.

Thanks guys.

Re: How to configure props.conf and transforms.conf to filter out a Windows eventcode?

pdjhh — Tue, 11 Aug 2015 12:42:31 GMT

Hi there. This did indeed work for me and I got my events down from 766 to about 170. Looking forward to seeing the license usage tomorrow. I had to do some new field extractions but the trimmed events were neat and tidy so easy to do. Thanks a lot.