Splunk Search

Is there an easy way to use blacklist in inputs.conf to filter the input itself in real-time like we would do for a Windows event log?

TobiasBoone
Communicator

Blacklisting works to blacklist a file or directory... but is there an easy way using blacklisting in inputs.conf to filter the input itself in real-time just like we would do from a Windows event log? To drop before indexing a Windows log, we would :

blacklist1 = EventCode="46**" Message="Account\sName:.*[\S\s]*Account\sName:\s+[\S+]+[\$]"
blacklist2 = EventCode="47**" Message="Logon\sAccount:.*[\S\s]*[dD]ocs_.*"

Is there a way to do something similar with a regular file input like in an IIS log to drop everything that ends in a .*\.jpg|png?

I know we can do it through props.confs and transforms files, but it would sure be easier to understand and deploy if only in the inputs.conf. The Windows event log filtering works very very well and would love to keep the configs similar.

0 Karma

lguinn2
Legend

Actually this is only possible for Event Log events - because Splunk has to parse them, even on a Universal Forwarder. Since Splunk has to translate the event log binary to text before it is forwarded, it seems logical to add the capability to filter the events locally.

However, no other data inputs are parsed during the inputs phase. Therefore, the events cannot be filtered using a Universal Forwarder.

If you must filter the events locally, you can use a Heavy Forwarder, which processes both the input phase and the parsing phase before forwarding. Be aware that this will increase the processing load on the local machine, which is usually a bad idea for a forwarder that lives on a production server.

TobiasBoone
Communicator

Obviously the forwarders have the capability to do regex on the event itself because the Windows Inputs using this method work like a champ. I just don't know if it is enabled on file inputs or what the syntax would be to specify looking at the lint itself and not the file.

0 Karma

somesoni2
Revered Legend

As far as I know, that is not possible (event level filter) with just inputs.conf. I would love to hear if anyone says otherwise.

Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...