How to blacklist events in WinEventLog that have a message containing quotes?

[dstaulcu]

I am playing with a new windows event log source called sysmon. Among other things, sysmon logs process creation events. In observation of process creation events, Splunk is by far the most noisy process on my test system. In trying to master advanced filtering techniques for wineventlogs, I figured I'd set about trying to blacklist any process create events supporting a splunk agent restart. I am having trouble getting a match on blacklist item #2 having quotes in match data. I've tried leaving the quotes unescaped and I have tried escaping the quotes using the backslash... Should this work? Is there another way?

Inputs appreciated:

[WinEventLog://Microsoft-Windows-Sysmon/Operational]
disabled = false
renderXml = false
# ignore process creates for process called by splunk.exe, splunkd.exe (nuclear option)
blacklist1=EventCode="1" Message="ParentImage: (C:\\Program Files\\SplunkUniversalForwarder\\bin\\splunkd.exe|C:\\Program Files\\SplunkUniversalForwarder\\bin\\splunk.exe)"
# ignore process creates for processes called by splunkd.exe restart task:   (restart scenario only)
blacklist2=EventCode="1" Message="ParentCommandLine: (\"C:\\Program Files\\SplunkUniversalForwarder\\bin\\splunkd.exe\" service)"

[jplumsdaine22]

Are you using a Universal Forwarder?; I don't think you can filter data with it. See http://docs.splunk.com/Documentation/Splunk/6.3.2/Forwarding/Routeandfilterdatad

[adambra]

Same question: I can't filter with a particular word in Message.

I wanted to filter the messages containing "Source Network Address:192.168.0.1" witch is the supervision server.

I tried (with and without backslashes):
blacklist = Message="/^.*192\.168\.0\.1.*$/m"
blacklist = Message="^.*192\.168\.0\.1.*$"
blacklist = Message="*192\.168\.0\.1*"
blacklist = Message="192\.168\.0\.1"

I tried a lot of expressions without success.

Any suggestion about that?

[dstaulcu]

I've been doing a lot of testing today and have an update on the scope of this problem.

Filter attempts (whitelist or blacklist) on EventCode and Message key value data appear to behave differently when renderXml = True compared to when renderXml = False.

Taking the following Event Message data for example:

fragment_begin<Data Name='Image'>C:\Windows\System32\ping.exe</Data><Data Name='CommandLine'>fragment_end

When renderXml = False, the following expression succeeds in filtering events:

blacklist = Message="<Data Name='Image'>.*\\(calc|ping).exe</Data><Data Name='CommandLine'>" 

However, when renderXml = True, the same expression fails to filter events.

After trying a various filtering strategies on this Message data when renderXml = True, it appears that matching fails any time when the < and > and , and = chars are included in the text to match. I've tried cancelling these characters with a backslash (or two or three) but no amount of that seems to work. I imagine there are other impacted characters as well.

Here are some of the testing patterns I have tried:

#Working
#blacklist = EventCode="1" Message="notepad.exe"
#blacklist = EventCode="1" Message="CommandLine"   
#blacklist = EventCode="1" Message=":"
#blacklist = EventCode="1" Message="\\"
#blacklist = EventCode="1" Message="(notepad.exe|calc.exe)"   
#blacklist = EventCode="1" Message="(\\notepad.exe|calc.exe)"   
#blacklist = EventCode="1" Message="(?i)(\\notePad.exe|calc.exe)" 
#blacklist = EventCode="1" Message="(?i)\\(ping.exe|calc.exe)" 
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe" 
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe.*"

#Not working
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe\<"
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe\<"
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe\\<"
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe\\\<"
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe\\\\<"
#blacklist = EventCode="1" Message="'CommandLine'"
#blacklist = EventCode="1" Message="'='"  
#blacklist = EventCode="1" Message="'\='"
#blacklist = EventCode="1" Message="\."

[nnmiller]

Please try using one of the following to work around this issue:

Unicode or ASCII version of special characters such as backslash, "<", or, ">". For example:
Unicode "\" = \u005c
Hex ASCII "\" = \x5c
Unicode "<" = u003c
Hex ASCII "<" = \x3c

Use {X} where "X" is the character count to indicate the number of occurrences of the character. E.g.:

#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe\<{1}"
#blacklist = EventCode="1" Message="\.{1}"

In SEDCMD these two methods are used as a workaround for oddities with the regex parser, so I am wondering if they work here as well.

[dstaulcu]

Thanks for the suggestion. Have to revisit this problem for another event type in today. Will give your suggestion a shot.

Meanwhile, here are some fill-in-the-blank blacklist patterns for all event types from Sysmon:

    # Sample blacklist entries for Sysmon schemaversion 4.21
blacklist1 = EventCode="^255$" Message="(?i)^UtcTime:\s+(.*)\s+ID:\s+(.*)\s+Description:\s+(.*)\s+Description:\s+(.*)$"

blacklist2 = EventCode="^1$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+FileVersion:\s+(.*)\s+Description:\s+(.*)\s+Product:\s+(.*)\s+Company:\s+(.*)\s+OriginalFileName:\s+(.*)\s+CommandLine:\s+(.*)\s+CurrentDirectory:\s+(.*)\s+User:\s+(.*)\s+LogonGuid:\s+(.*)\s+LogonId:\s+(.*)\s+TerminalSessionId:\s+(.*)\s+IntegrityLevel:\s+(.*)\s+Hashes:\s+(.*)\s+ParentProcessGuid:\s+(.*)\s+ParentProcessId:\s+(.*)\s+ParentImage:\s+(.*)\s+ParentCommandLine:\s+(.*)\s+ParentCommandLine:\s+(.*)$"

blacklist3 = EventCode="^2$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+TargetFilename:\s+(.*)\s+CreationUtcTime:\s+(.*)\s+PreviousCreationUtcTime:\s+(.*)\s+PreviousCreationUtcTime:\s+(.*)$"

blacklist4 = EventCode="^3$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+User:\s+(.*)\s+Protocol:\s+(.*)\s+Initiated:\s+(.*)\s+SourceIsIpv6:\s+(.*)\s+SourceIp:\s+(.*)\s+SourceHostname:\s+(.*)\s+SourcePort:\s+(.*)\s+SourcePortName:\s+(.*)\s+DestinationIsIpv6:\s+(.*)\s+DestinationIp:\s+(.*)\s+DestinationHostname:\s+(.*)\s+DestinationPort:\s+(.*)\s+DestinationPortName:\s+(.*)\s+DestinationPortName:\s+(.*)$"

blacklist5 = EventCode="^4$" Message="(?i)^UtcTime:\s+(.*)\s+State:\s+(.*)\s+Version:\s+(.*)\s+SchemaVersion:\s+(.*)\s+SchemaVersion:\s+(.*)$"

blacklist6 = EventCode="^5$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+Image:\s+(.*)$"

blacklist7 = EventCode="^6$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ImageLoaded:\s+(.*)\s+Hashes:\s+(.*)\s+Signed:\s+(.*)\s+Signature:\s+(.*)\s+SignatureStatus:\s+(.*)\s+SignatureStatus:\s+(.*)$"

blacklist8 = EventCode="^7$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+ImageLoaded:\s+(.*)\s+FileVersion:\s+(.*)\s+Description:\s+(.*)\s+Product:\s+(.*)\s+Company:\s+(.*)\s+OriginalFileName:\s+(.*)\s+Hashes:\s+(.*)\s+Signed:\s+(.*)\s+Signature:\s+(.*)\s+SignatureStatus:\s+(.*)\s+SignatureStatus:\s+(.*)$"

blacklist9 = EventCode="^8$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+SourceProcessGuid:\s+(.*)\s+SourceProcessId:\s+(.*)\s+SourceImage:\s+(.*)\s+TargetProcessGuid:\s+(.*)\s+TargetProcessId:\s+(.*)\s+TargetImage:\s+(.*)\s+NewThreadId:\s+(.*)\s+StartAddress:\s+(.*)\s+StartModule:\s+(.*)\s+StartFunction:\s+(.*)\s+StartFunction:\s+(.*)$"

blacklist10 = EventCode="^9$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+Device:\s+(.*)\s+Device:\s+(.*)$"

blacklist11 = EventCode="^10$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+SourceProcessGUID:\s+(.*)\s+SourceProcessId:\s+(.*)\s+SourceThreadId:\s+(.*)\s+SourceImage:\s+(.*)\s+TargetProcessGUID:\s+(.*)\s+TargetProcessId:\s+(.*)\s+TargetImage:\s+(.*)\s+GrantedAccess:\s+(.*)\s+CallTrace:\s+(.*)\s+CallTrace:\s+(.*)$"

blacklist12 = EventCode="^11$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+TargetFilename:\s+(.*)\s+CreationUtcTime:\s+(.*)\s+CreationUtcTime:\s+(.*)$"

blacklist13 = EventCode="^12$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+TargetObject:\s+(.*)\s+TargetObject:\s+(.*)$"

blacklist14 = EventCode="^13$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+TargetObject:\s+(.*)\s+Details:\s+(.*)\s+Details:\s+(.*)$"

blacklist15 = EventCode="^14$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+TargetObject:\s+(.*)\s+NewName:\s+(.*)\s+NewName:\s+(.*)$"

blacklist16 = EventCode="^15$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+TargetFilename:\s+(.*)\s+CreationUtcTime:\s+(.*)\s+Hash:\s+(.*)\s+Hash:\s+(.*)$"

blacklist17 = EventCode="^16$" Message="(?i)^UtcTime:\s+(.*)\s+Configuration:\s+(.*)\s+ConfigurationFileHash:\s+(.*)\s+ConfigurationFileHash:\s+(.*)$"

blacklist18 = EventCode="^17$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+PipeName:\s+(.*)\s+Image:\s+(.*)\s+Image:\s+(.*)$"

blacklist19 = EventCode="^18$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+PipeName:\s+(.*)\s+Image:\s+(.*)\s+Image:\s+(.*)$"

blacklist20 = EventCode="^19$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+Operation:\s+(.*)\s+User:\s+(.*)\s+EventNamespace:\s+(.*)\s+Name:\s+(.*)\s+Query:\s+(.*)\s+Query:\s+(.*)$"

blacklist21 = EventCode="^20$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+Operation:\s+(.*)\s+User:\s+(.*)\s+Name:\s+(.*)\s+Type:\s+(.*)\s+Destination:\s+(.*)\s+Destination:\s+(.*)$"

blacklist22 = EventCode="^21$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+Operation:\s+(.*)\s+User:\s+(.*)\s+Consumer:\s+(.*)\s+Filter:\s+(.*)\s+Filter:\s+(.*)$"

blacklist23 = EventCode="^22$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+QueryName:\s+(.*)\s+QueryStatus:\s+(.*)\s+QueryResults:\s+(.*)\s+Image:\s+(.*)\s+Image:\s+(.*)$"