inputs.conf blacklist regex for: Splunk_TA_microsoft-cloudservices

[gazoscreek]

Wondering if there's a blacklist parameter I can add to one of my Azure inputs so that Splunk will ignore pulling the event across the WAN. I already have a working ingest action, but the amount of data that's coming across is causing memory issues on my forwarder ...

my working ingest action is this ...
NETWORKSECURITYGROUPS\\/NSGBLAHBLAH.*?(IP\\.IP\\.IP\\.IP|IP\\.IP\\.IP\\.IP)

but is there an inputs.conf parameter I can set to this regex so that the data will be ignored at the source.

[gazoscreek]

Thank you for the quick and helpful reply. I figured that was probably the answer. In the meantime I'm working with the data owner at the origin to see if they can mitigate the issue on their end. Clearly something isn't right on the Azure client side and that'll need to be fixed. 

[livehybrid · Accepted solution]

Hi @gazoscreek 

No, the blacklist parameter in inputs.conf is not applicable for filtering event content collected by the Splunk_TA_microsoft-cloudservices add-on.

The blacklist parameter is used for file-based inputs (monitor, batch) to exclude files or directories based on their path. The Splunk_TA_microsoft-cloudservices collects data via APIs, not from files.

I believe you're stuck with the Index time parsing option which you are already looking at. Would you be able to share you config for this? We may be able to find some performance improvements which might help? Also, what is your architecture like? If there is too much pressure on your HF to do these parsings then are there other Intermediary forwarders that you could do it on, or perhaps even the indexers? This falls into the "it depends" category a little as I dont have all the info, but there may be some options out there.

Regarding the ingest_eval on another instance after the data has already been parsed on your HF, you can use RULESET- props.conf settings to call transforms - this is what Ingest Actions does to achieve transfoms on already parsed data.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing