Getting Data In

How do I filter out some Windows events at Search Head/Indexer (RHEL 6 install) Splunk 6.3.1?

New Member

New Splunk server, initial tuning period. Working on tuning and filtering. Server shows two event types as most frequent patterns:

44.49%  
12/09/2015 05:33:20 PM LogName=Security SourceName=Microsoft Windows security auditing. EventCode=5156 EventType=0 Type=Information ComputerName=LEE.cara.nascom.nasa.gov TaskCategory=Filtering Platform Connection OpCode=Info RecordNumber=70002132 Keywords=Audit Success Message=The Windows Filtering Platform has allowed a connection. Application Information: Process ID: 4 Application Name: System Network Information: Direction:  Inbound Source Address: xxx.xxx.xx.xxx Source Port: 138 Destination Address:    xxx.xxx.xx.xx Destination Port: 138 Protocol:   17 Filter Information: Filter Run-Time ID:  0 Layer Name:   Receive/Accept Layer Run-Time ID:   44

23.51%  
12/09/2015 05:30:38 PM LogName=Security SourceName=Microsoft Windows security auditing. EventCode=5158 EventType=0 Type=Information ComputerName=B28-WS71.cara.nascom.nasa.gov TaskCategory=Filtering Platform Connection OpCode=Info RecordNumber=1865188338 Keywords=Audit Success Message=The Windows Filtering Platform has permitted a bind to a local port. Application Information: Process ID:  13252 Application Name: \device\harddiskvolume2\program files\dell\dell data protection\access\advanced\wave\remotemanagement\wsceaa.exe Network Information: Source Address:   0.0.0.0 Source Port:    62358 Protocol: 6 Filter Information: Filter Run-Time ID:   0 Layer Name:   Resource Assignment Layer Run-Time ID:  36

Would like to filter those events on the Indexer so they are not ingested and don't count against indexing license cap. If possible would like to record first occurrence per day, then ignore duplicates. If that isn't possible, would be acceptable to filter all away for these specific events (at least in cases where audit was for a successful occurrence of an event)..

I've seen article here:
http://docs.splunk.com/Documentation/Splunk/6.3.1511/Forwarding/Routeandfilterdatad

But am still a little confused as to what needs to be done and where specifically to do same.

0 Karma

Splunk Employee
Splunk Employee

If you don't mind filtering at the source, you can use the blacklist feature on the windows event log modular input.

0 Karma

Splunk Employee
Splunk Employee

Why do it on the indexer, i.e. have the data be processed and forwarded over the network first, just to throw it away on the indexer, if you can do the same thing on the forwarder directly using inputs.conf eventID blacklisting?

0 Karma

New Member

That's easy for me to answer for my case - one configuration (on the indexer) versus multiple places to configure client systems that would be sending in the data. As long as the result is the same, I'd prefer to make the change in one place versus every client that I or someone else winds up deploying.

Also, given that more than myself may be deploying clients, if it is taken care of on the indexer I don't have to worry that one of my teammates did the install and missed the configuration and because of that we indexed too much data.

0 Karma

Splunk Employee
Splunk Employee

That tells me that you are not taking advantage of our Deployment Server to centrally manage forwarder configuration. I would suggest you consider that, if you can. It will also allow you to ensure that any changes made locally on the forwarder do not survive, as the deployment client will identify the deviation from what it should have and revert any changes. In the long run, you will be happier, I think.

0 Karma

SplunkTrust
SplunkTrust

Filtering is an all-or-none proposition - you can't keep the first event and filter the rest.

The props.conf and transforms.conf files will be in the 'local' directory for the app in which you are doing the filtering. If you don't have your own app, you'll probably want to modify the files for the search app.

NEVER modify a file in a default folder.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Splunk Employee
Splunk Employee

We don't support sampling, i.e. it's either all or nothing.
You can filter EventIDs in the inputs.conf of the source machine directly as described here and here using blacklisting of the IDs you don't want to index.

SplunkTrust
SplunkTrust

Which part of the "Filter event data and send to queues" section confuses you?

---
If this reply helps you, an upvote would be appreciated.
0 Karma

New Member

Where should I be looking for transforms.conf and props.conf that need to be updated?

I seem to recall, from prior work on another Splunk server that I'm supposed to be responsible for (but have limited experience in configuring) that there are more than one location where I might find those files and that one of the locations is a default area that may be replaced if the server is upgraded in the future.

I don't want to make the changes in the wrong place, and also am still not sure as to whether or not I could filter out only the later (2nd or later) occurrences of these events or just all of the events.

0 Karma