Is there an easy way to use blacklist in inputs.conf to filter the input itself in real-time like we would do for a Windows event log?

[TobiasBoone]

Blacklisting works to blacklist a file or directory... but is there an easy way using blacklisting in inputs.conf to filter the input itself in real-time just like we would do from a Windows event log? To drop before indexing a Windows log, we would :

blacklist1 = EventCode="46**" Message="Account\sName:.*[\S\s]*Account\sName:\s+[\S+]+[\$]"
blacklist2 = EventCode="47**" Message="Logon\sAccount:.*[\S\s]*[dD]ocs_.*"

Is there a way to do something similar with a regular file input like in an IIS log to drop everything that ends in a .*\.jpg|png?

I know we can do it through props.confs and transforms files, but it would sure be easier to understand and deploy if only in the inputs.conf. The Windows event log filtering works very very well and would love to keep the configs similar.

[lguinn2]

Actually this is only possible for Event Log events - because Splunk has to parse them, even on a Universal Forwarder. Since Splunk has to translate the event log binary to text before it is forwarded, it seems logical to add the capability to filter the events locally.

However, no other data inputs are parsed during the inputs phase. Therefore, the events cannot be filtered using a Universal Forwarder.

If you must filter the events locally, you can use a Heavy Forwarder, which processes both the input phase and the parsing phase before forwarding. Be aware that this will increase the processing load on the local machine, which is usually a bad idea for a forwarder that lives on a production server.

[TobiasBoone]

Obviously the forwarders have the capability to do regex on the event itself because the Windows Inputs using this method work like a champ. I just don't know if it is enabled on file inputs or what the syntax would be to specify looking at the lint itself and not the file.

[somesoni2]

As far as I know, that is not possible (event level filter) with just inputs.conf. I would love to hear if anyone says otherwise.