A lot of the Windows Security Events we see in Splunk, come from system-users that we're not interested in. I know there's a way to configure Splunk to filter out events based on the event content, but I can't get it to work.
In the example event below, I dont want to index events produced bt the
04/27/10 03:15:38 PM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4634
EventType=0
Type=Information
ComputerName=my.testmachine.env
TaskCategory=Logoff
OpCode=Info
RecordNumber=12345678
Keywords=Audit Success
Message=An account was logged off.
Subject:
Security ID: L-O-O-K-Y
Account Name: MAN-PAGE-H3LP$
Account Domain: TEST
Logon ID: 0x123456789
Logon Type: 7
My current configuration is -
inputs.conf:
[WinEventLog:Security]
disabled = 0
props.conf:
[source::WinEventLog:Security]
TRANSFORMS-Event=MachineNamePurge
blacklist = Account\sName:\s+\w+\$
I think you're just a bit unclear around the terminology & concepts in play here, we only implement 'blacklists' at the file level, not the event level. So you can blacklist entire files or directories using an entry in inputs.conf, but for individual events it's a bit more involved.
This would be more clear if you were familiar with the 3.x version of Splunk, back then it took both a props.conf entry AND a transforms.conf entry to extract a field from an event. With the current version you only need to do this if you are configuring an index-time field-extraction, which is basically what you're trying to achieve here.
In the config below, we have your regular inputs.conf entry -
Inputs.conf
[WinEventLog:Security]
disabled = 0
As you can see, nothing relating to blacklisting here. Next comes the props.conf entry -
Props.conf
[source::WinEventLog:Security]
TRANSFORMS-nullQ= nullFilter
and then the transforms.conf entry is where the actual REGEX and actions are defined -
Transforms.conf
[nullFilter]
REGEX = Account\sName:[\s|\w|-]+\$
DEST_KEY=queue
FORMAT = nullQueue
So in the 3.x world, if you had a setting in props.conf that started with 'REPORT-'. that meant it was a search-time field-extraction. If it started with 'TRANSFORMS-', that meant it was an index-time extraction. In this case, we want the action to run at index time, so we call it using TRANSFORMS.
In the transforms.conf entry, we apply the REGEX to every single event covered by the props.conf spec - WinEventLog:Security - so this is an expensive filtering method from a CPU perspective. If the REGEX matches, then the event is routed to the 'nullQueue', i.e. it's thrown away and never makes it into the index. This only affects events that match the REGEX, if it's not a match, then the event continues to the index as normal.
All of the configuration steps around this is documented here
While I have not been able to test this myself, the Splunk Cheatsheet (latest ver 2.21, Feb 2019) from MalwareArchaeology.com includes an example of how to do this in Splunk with some snippets here. Note that only 10 blacklists can be entered per
[WinEventLog://Security]
disabled=0
current_only=1
blacklist = 4689,5158
blacklist1 = EventCode="4688" Message="(?:New Process
Name:).+(?:SplunkUniversalForwarder\\bin\\splunk.exe)|.+(?:SplunkUniversalForwarder\\bin\\splunkd.exe)|.+(?:Splunk
UniversalForwarder\\bin\\btool.exe)"
...
blacklist4 = EventCode="4688" Message="(?:Process Command Line:).+(?:system32\\SearchFilterHost.exe)|.+(?:find
/i)|.+(?:Google\\Update\\GoogleUpdate.exe)|.+(?:WINDOWS\\system32\\conhost.exe)"
From the splunk docs on filtering Windows Events with inputs.conf (keywords: whitelist, blacklist), it notes:
You can have multiple key/regular expression sets in a single advanced filtering entry. Splunk software conjuncts the sets logically. This means that the entry is valid only if all of the sets in the entry are true.
You can specify up to 10 blacklists per stanza by adding a number to the end of the blacklist attribute, for example blacklist1...blacklist9.
Is it possible to blacklist an event from one index and whitelist same event for another index?
I think you're just a bit unclear around the terminology & concepts in play here, we only implement 'blacklists' at the file level, not the event level. So you can blacklist entire files or directories using an entry in inputs.conf, but for individual events it's a bit more involved.
This would be more clear if you were familiar with the 3.x version of Splunk, back then it took both a props.conf entry AND a transforms.conf entry to extract a field from an event. With the current version you only need to do this if you are configuring an index-time field-extraction, which is basically what you're trying to achieve here.
In the config below, we have your regular inputs.conf entry -
Inputs.conf
[WinEventLog:Security]
disabled = 0
As you can see, nothing relating to blacklisting here. Next comes the props.conf entry -
Props.conf
[source::WinEventLog:Security]
TRANSFORMS-nullQ= nullFilter
and then the transforms.conf entry is where the actual REGEX and actions are defined -
Transforms.conf
[nullFilter]
REGEX = Account\sName:[\s|\w|-]+\$
DEST_KEY=queue
FORMAT = nullQueue
So in the 3.x world, if you had a setting in props.conf that started with 'REPORT-'. that meant it was a search-time field-extraction. If it started with 'TRANSFORMS-', that meant it was an index-time extraction. In this case, we want the action to run at index time, so we call it using TRANSFORMS.
In the transforms.conf entry, we apply the REGEX to every single event covered by the props.conf spec - WinEventLog:Security - so this is an expensive filtering method from a CPU perspective. If the REGEX matches, then the event is routed to the 'nullQueue', i.e. it's thrown away and never makes it into the index. This only affects events that match the REGEX, if it's not a match, then the event continues to the index as normal.
All of the configuration steps around this is documented here