Getting Data In

Blacklist files by content

jdmeek
Explorer

We have all of our Windows workstations sending PowerShell transcripts to a central drop share for ingest.  This worked well until our endpoint group recently started moving things to InTune.  Now we're getting hammered by InTune generated PS scripts and need to blacklist them.

The problem is that the files from InTune are the same naming scheme as the transcripts that we want to keep and I've not found a way to alter the naming scheme so I can't blacklist them by filename or path. 

The files contain script path info that I can filter on but that does me no good if I want to ignore the whole file.  

So is there a way to blacklist/ignore an entire file based on a regex match of the contents?

0 Karma

livehybrid
SplunkTrust
SplunkTrust

Hi @jdmeek 

blacklist/whitelist in inputs.conf only match against the file path/name, not content, so there's no monitor-stanza setting that can exclude a whole file based on what's inside it (see inputs.conf include/exclude docs).

Given your PowerShell transcripts are indexed as a single multi-line event per file, you can drop the whole thing at parse time instead, by matching a string that only appears in the InTune-generated transcripts and routing that event to the null queue before indexing:

#props.conf
[your_sourcetype]
TRANSFORMS-dropintune = drop_intune_transcripts

#transforms.conf
[drop_intune_transcripts]
REGEX = (?i)Microsoft.Management.Services.IntuneWindowsAgent
DEST_KEY = queue
FORMAT = nullQueue

 

This has to run on the forwarder/HF that runs the parsing pipeline, since nullQueue must be applied before the event hits the indexing queue. Use a REGEX string that's unique to the InTune path info inside the file (e.g. a script path fragment you mentioned you can already filter on) rather than anything generic, otherwise you'll null-queue legitimate transcripts too. If the transcript header/footer is consistent enough, matching against it is more reliable than matching mid-script content.

🌟 Did this answer help you? If so, please consider:

    • Adding karma to show it was useful
    • Marking it as the solution if it resolved your issue
    • Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing.

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Short answer is no. There is no built-in functionality which does ingestion of a whole file depending on its contents. You could try to tackle this problem by using your own script to move files based on contents into a "spool" directory for ingestion.

Or by writing your own scripted input (assuming it's a UF) which would selectively read files. But that would mean handling checkpoints and such; it's not a quick and dirty one-liner.

If the contents on the other hand are distinguishable on a per-event basis, you could try filtering them out after initial pickup by the input (if you want to save on bandwidth you could enable additional processing on the UF).

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Watch On-Demand   Replay the Accelerate ...

Splunk Agent Observability: Measuring AI Value with Splunk Tokenomics on AWS

The AI landscape is moving faster than ever, and our latest 2026 State of Observability report provides the ...

Enterprise Security Watch Party: What’s New in 8.8 - Recap

Splunk Community HQ hosted a Splunk Enterprise Security Watch Party: What’s New In ES 8.8 last Thursday. I had ...