Filtering with props and transforms

michaelnorup — Thu, 18 Nov 2021 07:10:31 GMT

Hello.

I have just enabled powershell logging and am now getting completely spammed with splunk forwarders running powershell scripts, so i am trying to filter that out.

It is a distributed setup, with:

1 deployment/license server
1 indexer
1 search head

On the indexer i have added

splunk/etc/apps/windowsServers/local/props.conf

[WinEventLog:Microsoft-Windows-Powershell/Operational] SHOULD_LINEMERG = false MAX_TIMESTAP_LOOKAHEAD=300 detect_trailing_nulls = auto TIME_PREFIX = "\clock\": KV_MODE = json AUTO_KV_JSON = true TRANSFORMS-set = setnull

and transforms.conf

[setnull] REGEX = (?mi)SplunkUniversalForwarder DEST_KEY = queue FORMAT = nullQueue

On the deployment/license server i have added the same transforms.conf as above

On the Search head i have added props.conf:

[WinEventLog:Microsoft-Windows-Powershell/Operational] SHOULD_LINEMERGE = false detect_trailing_nulls = auto KV_MODE = json AUTO_KV_JSON = true

A log entry that i would like filtered out could look like this:

11/16/2021 01:38:27 PM LogName=Microsoft-Windows-PowerShell/Operational SourceName=Microsoft-Windows-PowerShell EventCode=4103 EventType=4 Type=Information ComputerName=********* User=SYSTEM Sid=S-1-5-18 SidType=1 TaskCategory=Executing Pipeline OpCode=To be used when operation is just executing a method RecordNumber=16918098 Keywords=None Message=CommandInvocation(Select-Object): "Select-Object" ParameterBinding(Select-Object): name="ExpandProperty"; value="UserProcessorTime" ParameterBinding(Select-Object): name="InputObject"; value="System.Diagnostics.Process (WmiPrvSE)" Context: Severity = Informational Host Name = ConsoleHost Host Version = 4.0 Host ID = c939e89c-c7f2-468c-b351-295154efdf43 Host Application = powershell.exe -command & {get-content C:\Windows\TEMP\\input51852c9500841041.tmp | c:\Program` Files\SplunkUniversalForwarder\bin\splunk-powershell.ps1 c:\Program` Files\SplunkUniversalForwarder 51852c9500841041} Engine Version = 4.0 Runspace ID = bb969a43-cc4d-4f2c-b4ec-80b7ed2ff7d7 Pipeline ID = 1 Command Name = Select-Object Command Type = Cmdlet Script Name = C:\Program Files\SplunkUniversalForwarder\bin\splunk-powershell-common.ps1 Command Path = Sequence Number = 104739 User = ****** Shell ID = Microsoft.PowerShell User Data:

They all contain the SplunkUniversalForwarder which is why i tried to use that in the regex

When i add all these things, all logs from the sourcetype disappear. What am i doing wrong?

Re: Filtering with props and transforms

vhharanpositka — Tue, 16 Nov 2021 13:52:55 GMT

Hi @michaelnorup

Based on my understanding, the regex you are using is matching all the events.
So you need to use any unique value in the regex.

Ex. Eventcode.

Else you can disable this input

[WinEventLog:Microsoft-Windows-Powershell/Operational]

disabled = 1

Re: Filtering with props and transforms

michaelnorup — Tue, 16 Nov 2021 13:57:28 GMT

The regex could be wrong, but i dont think thats the issue. The log here is evetncode 4103, which is the ones i am actually interested in. Problem is that it looks like the Universalforwarder runs a powershell script constantly, and it is spamming my data, so i need to filter that single script out, and keep the rest of the powershell logs.

topic Re: Filtering with props and transforms in Getting Data In

Filtering with props and transforms

Re: Filtering with props and transforms

Re: Filtering with props and transforms