I am ingesting Windows event logs including event code 5156 which is chewing up a lot of license. I have had it turned off (as it is 'firewall permitted connection'), but we want to track connections from these hosts to external IP addresses so I have switched it back on. What I'm thinking is, it would be good to be able to filter out all of the events that are destined for the internal network as they are not of interest.
The internal network is 192.168.x.x so I'd be looking to filter out all of those and let the rest get indexed. That would mean filtering out EventCode=4624 with field Destination Address=192.168*.
Looking around this will probably be a props and transforms with regex or potentially a fancy blacklist. Hope you can advise.
Hi there. This did indeed work for me and I got my events down from 766 to about 170. Looking forward to seeing the license usage tomorrow. I had to do some new field extractions but the trimmed events were neat and tidy so easy to do. Thanks a lot.
I'd say you need a black- and then a whitelist, so something like this in transforms.conf:
[throwout] REGEX = (?m)^EventCode=(5156) DEST_KEY = queue FORMAT = nullQueue [keepsome] REGEX = Destination\sAddress\:\s192\.168\. DEST_KEY = queue FORMAT = indexQueue
and this in props.conf:
[WinEventLog:Security] TRANSFORMS-set= throwout,keepsome
PS: I just realized that the above does the opposite of what you intended to do - it throws out all allowed firewall events, and it keeps specifically those destined for your local network. I must've misread your question halfway through, sorry.
One possibility to achieve your goal is this, where we assume there are no other occurences of "Destination Adress = some ip" in any of the windows events (which I'm not really sure about, but preliminary googling revealed these to be most common in events 5156 and 5152). Props.conf:
[WinEventLog:Security] TRANSFORMS-set= throwoutlocal
[throwoutlocal] REGEX = Destination\sAddress\:\s192\.168\. DEST_KEY = queue FORMAT = nullQueue
A more reliable solution would be to use the cool method runals provided in the other answer - you won't miss any events at all, and you'll still cut down on the license and storage capacity for those prattling windows event logs.
Thanks for getting back to me. I'm very new to regex but that looks like you're keeping the events with 192.168. in them? I need to drop those ones and keep the rest. What do you think?
08/11/2015 05:48:23 PM
SourceName=Microsoft Windows security auditing.
TaskCategory=Filtering Platform Connection
Message=The Windows Filtering Platform has permitted a connection.
Process ID: 1628
Application Name: \device\harddiskvolume2\program files\vmware\vmware view\agent\bin\wsnm_jms.exe
Source Address: 192.168.10.200
Source Port: 49159
Destination Address: 192.168.6.21
Destination Port: 4001
Filter Run-Time ID: 76977
Layer Name: Connect
Layer Run-Time ID: 48
I had the .conf files in the wrong location on the forwarder! Moved them to etc/system/local and started to take effect. Based on that I moved onto the solution mentioned by runals below which also worked after some addtional field extractions I had to do on the newly trimmed events.
IT could be working now I'm not sure as I'm not getting any hits. The connections to non 192 addresses are few and far between so I'll leave it run for a while. If I see any events destined for non 192 addresses then we must be there..
I'm seeing a huge number less events but still ones with 192 destination addresses. Problem with that is I don't really know what's working and what isn't. Will need to look deeper. Thanks.