How to create a regex for these events?

[AL3Z]

Hi All,

Can any one pls share a regex for the below events to exclude(text in red).

1.
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{5484D}'/><EventID>4688</EventID><Version>2</Version><Level>0</Level><Task>13312</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime='2023-09-26T18:27:56.545195800Z'/><EventRecordID>2371</EventRecordID><Correlation/><Execution ProcessID='4' ThreadID='18656'/><Channel>Security</Channel><Computer>securejump</Computer><Security/></System><EventData><Data Name='SubjectUserSid'>NT AUTHORITY\SYSTEM</Data><Data Name='SubjectUserName'>SECUREJUMP</Data><Data Name='SubjectDomainName'>EC</Data><Data Name='SubjectLogonId'>0x37</Data><Data Name='NewProcessId'>0x140</Data><Data Name='NewProcessName'>C:\Program Files\Windows Defender Advanced Threat Protection\SenseCncProxy.exe</Data><Data Name='TokenElevationType'>%j1936</Data><Data Name='ProcessId'>0x3520</Data><Data Name='CommandLine'></Data><Data Name='TargetUserSid'>NULL SID</Data><Data Name='TargetUserName'>-</Data><Data Name='TargetDomainName'>-</Data><Data Name='TargetLogonId'>0x0</Data><Data Name='ParentProcessName'>C:\Program Files\Windows Defender Advanced Threat Protection\MsSense.exe</Data><Data Name='MandatoryLabel'>Mandatory Label\System Mandatory Level</Data></EventData></Event>

2.
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{hh}'/><EventID>4688</EventID><Version>2</Version><Level>0</Level><Task>13312</Task><Opcode>0</Opcode><Keywords>0000000</Keywords><TimeCreated SystemTime='2023-09-26T18:00:46.762007500Z'/><EventRecordID>146821602</EventRecordID><Correlation/><Execution ProcessID='4' ThreadID='24996'/><Channel>Security</Channel><Computer>securejump</Computer><Security/></System><EventData><Data Name='SubjectUserSid'>NT AUTHORITY\SYSTEM</Data><Data Name='SubjectUserName'>SECUREJUMP</Data><Data Name='SubjectDomainName'>EC</Data><Data Name='SubjectLogonId'>03e7</Data><Data Name='NewProcessId'>0511c</Data><Data Name='NewProcessName'>C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe</Data><Data Name='TokenElevationType'>%%1936</Data><Data Name='ProcessId'>0x2010</Data><Data Name='CommandLine'></Data><Data Name='TargetUserSid'>NULL SID</Data><Data Name='TargetUserName'>-</Data><Data Name='TargetDomainName'>-</Data><Data Name='TargetLogonId'>0x0</Data><Data Name='ParentProcessName'>C:\Program Files\Windows Defender Advanced Threat Protection\SenseIR.exe</Data><Data Name='MandatoryLabel'>Mandatory Label\System Mandatory Level</Data></EventData></Event>

Need a single regex to exclude 1& 2 events.

<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{54849625'/><EventID>4688</EventID><Version>2</Version><Level>0</Level><Task>13312</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime='2023-09-26T17:44:16.666598900Z'/><EventRecordID>146821089</EventRecordID><Correlation/><Execution ProcessID='4' ThreadID='2136'/><Channel>Security</Channel><Computer>secu</Computer><Security/></System><EventData><Data Name='SubjectUserSid'>NT AUTHORITY\SYSTEM</Data><Data Name='SubjectUserName'>SEC</Data><Data Name='SubjectDomainName'>EC</Data><Data Name='SubjectLogonId'>0x3e7</Data><Data Name='NewProcessId'>0x51</Data><Data Name='NewProcessName'>C:\Windows\System32\conhost.exe</Data><Data Name='TokenElevationType'>%%1936</Data><Data Name='ProcessId'>0x3ec</Data><Data Name='CommandLine'></Data><Data Name='TargetUserSid'>NULL SID</Data><Data Name='TargetUserName'>-</Data><Data Name='TargetDomainName'>-</Data><Data Name='TargetLogonId'>0x0</Data><Data Name='ParentProcessName'>C:\Program Files\AzureConnectedMachineAgent\GCArcService\GC\gc_worker.exe</Data><Data Name='MandatoryLabel'>Mandatory Label\System Mandatory Level</Data></EventData></Event>

<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{449'/><EventID>4688</EventID><Version>2</Version><Level>0</Level><Task>13312</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime='2023-09-26T18:24:19.611633300Z'/><EventRecordID>146822267</EventRecordID><Correlation/><Execution ProcessID='4' ThreadID='19952'/><Channel>Security</Channel><Computer>securejump</Computer><Security/></System><EventData><Data Name='SubjectUserSid'>NT AUTHORITY\SYSTEM</Data><Data Name='SubjectUserName'>SECUREJUMP</Data><Data Name='SubjectDomainName'>EC</Data><Data Name='SubjectLogonId'>0x3e7</Data><Data Name='NewProcessId'>0x4a18</Data><Data Name='NewProcessName'>C:\Program Files\Rapid7\Insight Agent\components\insight_agent\3.2.5.31\get_proxy.exe</Data><Data Name='TokenElevationType'>%%1936</Data><Data Name='ProcessId'>0xdd0</Data><Data Name='CommandLine'></Data><Data Name='TargetUserSid'>NULL SID</Data><Data Name='TargetUserName'>-</Data><Data Name='TargetDomainName'>-</Data><Data Name='TargetLogonId'>0x0</Data><Data Name='ParentProcessName'>C:\Program Files\Rapid7\Insight Agent\components\insight_agent\3.2.5.31\ir_agent.exe</Data><Data Name='MandatoryLabel'>Mandatory Label\System Mandatory Level</Data></EventData></Event>

Thanks...

[PickleRick]

What do you mean by "exclude" here? You want to blacklist them on input or exclude them from search results? And why would you need a single regex to match dwo different patterns?

You put this post in "deployment architecture" section when it has nothing to do with architecture and tagged it with "deployment server" which again it has nothing to do with. So what is it about?

[AL3Z]

hi 

Want to blacklist them on inputs as I left with only three  blacklist space.

[PickleRick]

OK. That makes sense. The number of remaining blacklist entries is a valid point.

You can construct a regex matchig several partially alternative branches using the A|B construct.

Also remember that while using renderXml=true you need to blacklist by $XmlRegex field.

So you'd end up with something like this:

blacklist9 = $XmlRegex=#Data Name='ParentProcessName'>C:\\Program Files\\(AzureConnectedMachineAgent\\GCArcService\\GC\\gc_worker\.exe|Rapid7\\Insight Agent\\components\\insight_agent\\3\.2\.5\.31\\ir_agent\.exe)#

You might need to escape the > sign (I never remember which solutions treated the raw '>' as literal '>' and escaped '\>' as end of the word. And which ones did the opposite. (I think vim was notorious for strangely (un)escaped characters inregexes).

As usual - try your regex at regex101.com

Of course if you wanted, you can combine all your 4 cases into one using the alternative grouping.

[AL3Z]

@PickleRick ,

Hello, When I apply this blacklist  regex, still I can see the logs. Can we use btool to trouble shoot this issue ??
blacklist8 = "$XmlRegex=#Data Name='ParentProcessName'>C:\\Program Files\\(AzureConnectedMachineAgent\\GCArcService\\GC\\(gc_service|gc_worker)\.exe|Windows Defender Advanced Threat Protection\\(MsSense|SenseCM|SenseIR)\.exe|Rapid7\\Insight Agent\\components\\insight_agent\\3\.2\.5\.31\\ir_agent\.exe)#"
renderXml=true

Thanks 

[PickleRick]

Regexes in blacklists can be tricky sometimes. btool will just show you what is the effective config you just wrote so it won't show you if it works or not.

I assume you restarted your forwarder after configuring the blacklist.

Anyway, you should not enclose the blacklist parameter in quotes.

[AL3Z]

Yes I had restarted forwarder but in the host  inputs.conf I dnt see the applied regex from deployment server !

As we are using all the blacklisted in the quotes!!

[PickleRick]

Wait. What do you mean? You're editing the app on DS? Then you have to reload the deployment server (you don't have to restart it) so that it notices a new version of the app and offers it to the forwarder(s) for download.

Also:

https://docs.splunk.com/Documentation/Splunk/9.1.1/Admin/Inputsconf#Event_Log_allow_list_and_deny_li...

[AL3Z]

Hi,
@PickleRick @gcusello 

When I tried to refresh/debug inputs.conf using btool in deployment server , I can see the below errors.

Refreshing admin/collections-conf
RESTException [HTTP 503] [{'type': 'ERROR', 'code': None, 'text': 'KV Store initialization failed.
Please contact your system administrator.'}]

Refreshing admin/deploymentserver SplunkdConnectionException Splunkd daemon is not responding: ('Error connecting to /servicesNS/nobody/search/admin/deploymentserver/_reload: The read operation timed out',)

Refreshing admin/ingest-rfs-destinations SplunkdConnectionException Splunkd daemon is not responding: ('Error connecting to /servicesNS/nobody/search/admin/ingest-rfs-destinations/_reload: The read operation timed out',)

Refreshing admin/serverclasses SplunkdConnectionException Splunkd daemon is not responding: ('Error connecting to /servicesNS/nobody/search/admin/serverclasses/_reload: The read operation timed out',)

How we can trouble shoot this ERROR Messages ??

[gcusello]

Hi @AL3Z ,

do you want to remove all the events from the input or only the selectes part of the events?

If you want to remove all the events, you could use a simple regex to blacklist them:

Data Name\=\'ParentProcessName\'\>C:\\Program Files\\(Windows Defender Advanced Threat Protection\\MsSense\.exe)|(Windows Defender Advanced Threat Protection\\SenseIR\.exe)|(AzureConnectedMachineAgent\\GCArcService\\GC\\gc_worker\.exe)|(Rapid7\\Insight Agent\\components\\insight_agent\\3\.2\.5\.31\\ir_agent\.exe)

you can check this regex at https://regex101.com/r/9lsjyz/1. Ciao.

Giuseppe

[AL3Z]

Hi @gcusello ,

only the selected part of the events i am trying to exclude..
How we can trouble shoot splunk locally using btool ?

[gcusello]

Hi @AL3Z,

if you want to remove only a part of events, you have to follow the instructions at https://docs.splunk.com/Documentation/Splunk/9.1.1/Data/Anonymizedata

you should insert in your props.conf  one SEDCMD-<class> = y/<string1>/<string2>/g, using my above regex:

SEDCMD-remove_strings = s/Data Name\=\'ParentProcessName\'\>C:\\Program Files\\(Windows Defender Advanced Threat Protection\\MsSense\.exe)|(Windows Defender Advanced Threat Protection\\SenseIR\.exe)|(AzureConnectedMachineAgent\\GCArcService\\GC\\gc_worker\.exe)|(Rapid7\\Insight Agent\\components\\insight_agent\\3\.2\.5\.31\\ir_agent\.exe)//g

 Ciao.

Giuseppe