We have all of our Windows workstations sending PowerShell transcripts to a central drop share for ingest. This worked well until our endpoint group recently started moving things to InTune. Now we're getting hammered by InTune generated PS scripts and need to blacklist them.
The problem is that the files from InTune are the same naming scheme as the transcripts that we want to keep and I've not found a way to alter the naming scheme so I can't blacklist them by filename or path.
The files contain script path info that I can filter on but that does me no good if I want to ignore the whole file.
So is there a way to blacklist/ignore an entire file based on a regex match of the contents?
Hi @jdmeek
blacklist/whitelist in inputs.conf only match against the file path/name, not content, so there's no monitor-stanza setting that can exclude a whole file based on what's inside it (see inputs.conf include/exclude docs).
Given your PowerShell transcripts are indexed as a single multi-line event per file, you can drop the whole thing at parse time instead, by matching a string that only appears in the InTune-generated transcripts and routing that event to the null queue before indexing:
#props.conf
[your_sourcetype]
TRANSFORMS-dropintune = drop_intune_transcripts
#transforms.conf
[drop_intune_transcripts]
REGEX = (?i)Microsoft.Management.Services.IntuneWindowsAgent
DEST_KEY = queue
FORMAT = nullQueue
This has to run on the forwarder/HF that runs the parsing pipeline, since nullQueue must be applied before the event hits the indexing queue. Use a REGEX string that's unique to the InTune path info inside the file (e.g. a script path fragment you mentioned you can already filter on) rather than anything generic, otherwise you'll null-queue legitimate transcripts too. If the transcript header/footer is consistent enough, matching against it is more reliable than matching mid-script content.
🌟 Did this answer help you? If so, please consider:
Your feedback encourages the volunteers in this community to continue contributing.
Short answer is no. There is no built-in functionality which does ingestion of a whole file depending on its contents. You could try to tackle this problem by using your own script to move files based on contents into a "spool" directory for ingestion.
Or by writing your own scripted input (assuming it's a UF) which would selectively read files. But that would mean handling checkpoints and such; it's not a quick and dirty one-liner.
If the contents on the other hand are distinguishable on a per-event basis, you could try filtering them out after initial pickup by the input (if you want to save on bandwidth you could enable additional processing on the UF).