topic Re: Need a regex for these events in Getting Data In

How to create a regex for these events?

Raj — Wed, 27 Sep 2023 19:08:46 GMT

Hi All,

Can any one pls share a regex for the below events to exclude(text in red).

1.
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{5484D}'/><EventID>4688</EventID><Version>2</Version><Level>0</Level><Task>13312</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime='2023-09-26T18:27:56.545195800Z'/><EventRecordID>2371</EventRecordID><Correlation/><Execution ProcessID='4' ThreadID='18656'/><Channel>Security</Channel><Computer>securejump</Computer><Security/></System><EventData><Data Name='SubjectUserSid'>NT AUTHORITY\SYSTEM</Data><Data Name='SubjectUserName'>SECUREJUMP</Data><Data Name='SubjectDomainName'>EC</Data><Data Name='SubjectLogonId'>0x37</Data><Data Name='NewProcessId'>0x140</Data><Data Name='NewProcessName'>C:\Program Files\Windows Defender Advanced Threat Protection\SenseCncProxy.exe</Data><Data Name='TokenElevationType'>%j1936</Data><Data Name='ProcessId'>0x3520</Data><Data Name='CommandLine'></Data><Data Name='TargetUserSid'>NULL SID</Data><Data Name='TargetUserName'>-</Data><Data Name='TargetDomainName'>-</Data><Data Name='TargetLogonId'>0x0</Data><Data Name='ParentProcessName'>C:\Program Files\Windows Defender Advanced Threat Protection\MsSense.exe</Data><Data Name='MandatoryLabel'>Mandatory Label\System Mandatory Level</Data></EventData></Event>

2.
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{hh}'/><EventID>4688</EventID><Version>2</Version><Level>0</Level><Task>13312</Task><Opcode>0</Opcode><Keywords>0000000</Keywords><TimeCreated SystemTime='2023-09-26T18:00:46.762007500Z'/><EventRecordID>146821602</EventRecordID><Correlation/><Execution ProcessID='4' ThreadID='24996'/><Channel>Security</Channel><Computer>securejump</Computer><Security/></System><EventData><Data Name='SubjectUserSid'>NT AUTHORITY\SYSTEM</Data><Data Name='SubjectUserName'>SECUREJUMP</Data><Data Name='SubjectDomainName'>EC</Data><Data Name='SubjectLogonId'>03e7</Data><Data Name='NewProcessId'>0511c</Data><Data Name='NewProcessName'>C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe</Data><Data Name='TokenElevationType'>%%1936</Data><Data Name='ProcessId'>0x2010</Data><Data Name='CommandLine'></Data><Data Name='TargetUserSid'>NULL SID</Data><Data Name='TargetUserName'>-</Data><Data Name='TargetDomainName'>-</Data><Data Name='TargetLogonId'>0x0</Data><Data Name='ParentProcessName'>C:\Program Files\Windows Defender Advanced Threat Protection\SenseIR.exe</Data><Data Name='MandatoryLabel'>Mandatory Label\System Mandatory Level</Data></EventData></Event>

Need a single regex to exclude 1& 2 events.

<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{54849625'/><EventID>4688</EventID><Version>2</Version><Level>0</Level><Task>13312</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime='2023-09-26T17:44:16.666598900Z'/><EventRecordID>146821089</EventRecordID><Correlation/><Execution ProcessID='4' ThreadID='2136'/><Channel>Security</Channel><Computer>secu</Computer><Security/></System><EventData><Data Name='SubjectUserSid'>NT AUTHORITY\SYSTEM</Data><Data Name='SubjectUserName'>SEC</Data><Data Name='SubjectDomainName'>EC</Data><Data Name='SubjectLogonId'>0x3e7</Data><Data Name='NewProcessId'>0x51</Data><Data Name='NewProcessName'>C:\Windows\System32\conhost.exe</Data><Data Name='TokenElevationType'>%%1936</Data><Data Name='ProcessId'>0x3ec</Data><Data Name='CommandLine'></Data><Data Name='TargetUserSid'>NULL SID</Data><Data Name='TargetUserName'>-</Data><Data Name='TargetDomainName'>-</Data><Data Name='TargetLogonId'>0x0</Data><Data Name='ParentProcessName'>C:\Program Files\AzureConnectedMachineAgent\GCArcService\GC\gc_worker.exe</Data><Data Name='MandatoryLabel'>Mandatory Label\System Mandatory Level</Data></EventData></Event>

<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{449'/><EventID>4688</EventID><Version>2</Version><Level>0</Level><Task>13312</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime='2023-09-26T18:24:19.611633300Z'/><EventRecordID>146822267</EventRecordID><Correlation/><Execution ProcessID='4' ThreadID='19952'/><Channel>Security</Channel><Computer>securejump</Computer><Security/></System><EventData><Data Name='SubjectUserSid'>NT AUTHORITY\SYSTEM</Data><Data Name='SubjectUserName'>SECUREJUMP</Data><Data Name='SubjectDomainName'>EC</Data><Data Name='SubjectLogonId'>0x3e7</Data><Data Name='NewProcessId'>0x4a18</Data><Data Name='NewProcessName'>C:\Program Files\Rapid7\Insight Agent\components\insight_agent\3.2.5.31\get_proxy.exe</Data><Data Name='TokenElevationType'>%%1936</Data><Data Name='ProcessId'>0xdd0</Data><Data Name='CommandLine'></Data><Data Name='TargetUserSid'>NULL SID</Data><Data Name='TargetUserName'>-</Data><Data Name='TargetDomainName'>-</Data><Data Name='TargetLogonId'>0x0</Data><Data Name='ParentProcessName'>C:\Program Files\Rapid7\Insight Agent\components\insight_agent\3.2.5.31\ir_agent.exe</Data><Data Name='MandatoryLabel'>Mandatory Label\System Mandatory Level</Data></EventData></Event>

Thanks...

Re: Need a regex for these events

PickleRick — Tue, 26 Sep 2023 19:00:27 GMT

What do you mean by "exclude" here? You want to blacklist them on input or exclude them from search results? And why would you need a single regex to match dwo different patterns?

You put this post in "deployment architecture" section when it has nothing to do with architecture and tagged it with "deployment server" which again it has nothing to do with. So what is it about?

Re: Need a regex for these events

Raj — Tue, 26 Sep 2023 19:56:26 GMT

Want to blacklist them on inputs as I left with only three blacklist space.

Re: Need a regex for these events

gcusello — Wed, 27 Sep 2023 06:56:31 GMT

Hi @Raj ,

do you want to remove all the events from the input or only the selectes part of the events?

If you want to remove all the events, you could use a simple regex to blacklist them:

Data Name\=\'ParentProcessName\'\>C:\\Program Files\\(Windows Defender Advanced Threat Protection\\MsSense\.exe)|(Windows Defender Advanced Threat Protection\\SenseIR\.exe)|(AzureConnectedMachineAgent\\GCArcService\\GC\\gc_worker\.exe)|(Rapid7\\Insight Agent\\components\\insight_agent\\3\.2\.5\.31\\ir_agent\.exe)

you can check this regex at https://regex101.com/r/9lsjyz/1. Ciao.

Giuseppe

Re: Need a regex for these events

PickleRick — Wed, 27 Sep 2023 07:23:23 GMT

OK. That makes sense. The number of remaining blacklist entries is a valid point.

You can construct a regex matchig several partially alternative branches using the A|B construct.

Also remember that while using renderXml=true you need to blacklist by $XmlRegex field.

So you'd end up with something like this:

blacklist9 = $XmlRegex=#Data Name='ParentProcessName'>C:\\Program Files\\(AzureConnectedMachineAgent\\GCArcService\\GC\\gc_worker\.exe|Rapid7\\Insight Agent\\components\\insight_agent\\3\.2\.5\.31\\ir_agent\.exe)#

You might need to escape the > sign (I never remember which solutions treated the raw '>' as literal '>' and escaped '\>' as end of the word. And which ones did the opposite. (I think vim was notorious for strangely (un)escaped characters inregexes).

As usual - try your regex at regex101.com

Of course if you wanted, you can combine all your 4 cases into one using the alternative grouping.

Re: Need a regex for these events

Raj — Wed, 27 Sep 2023 09:59:28 GMT

@PickleRick ,

Hello, When I apply this blacklist regex, still I can see the logs. Can we use btool to trouble shoot this issue ??
blacklist8 = "$XmlRegex=#Data Name='ParentProcessName'>C:\\Program Files\\(AzureConnectedMachineAgent\\GCArcService\\GC\\(gc_service|gc_worker)\.exe|Windows Defender Advanced Threat Protection\\(MsSense|SenseCM|SenseIR)\.exe|Rapid7\\Insight Agent\\components\\insight_agent\\3\.2\.5\.31\\ir_agent\.exe)#"
renderXml=true

Thanks

Re: Need a regex for these events

Raj — Wed, 27 Sep 2023 11:17:35 GMT

Hi @gcusello ,

only the selected part of the events i am trying to exclude..
How we can trouble shoot splunk locally using btool ?

Re: Need a regex for these events

PickleRick — Wed, 27 Sep 2023 11:26:15 GMT

Regexes in blacklists can be tricky sometimes. btool will just show you what is the effective config you just wrote so it won't show you if it works or not.

I assume you restarted your forwarder after configuring the blacklist.

Anyway, you should not enclose the blacklist parameter in quotes.

Re: Need a regex for these events

Raj — Wed, 27 Sep 2023 11:45:22 GMT

Yes I had restarted forwarder but in the host inputs.conf I dnt see the applied regex from deployment server !

As we are using all the blacklisted in the quotes!!

Re: Need a regex for these events

PickleRick — Wed, 27 Sep 2023 11:53:03 GMT

Wait. What do you mean? You're editing the app on DS? Then you have to reload the deployment server (you don't have to restart it) so that it notices a new version of the app and offers it to the forwarder(s) for download.

Also:

https://docs.splunk.com/Documentation/Splunk/9.1.1/Admin/Inputsconf#Event_Log_allow_list_and_deny_list_formats

Re: Need a regex for these events

Raj — Thu, 28 Sep 2023 12:07:57 GMT

Hi,
@PickleRick @gcusello

When I tried to refresh/debug inputs.conf using btool in deployment server , I can see the below errors.

Refreshing admin/collections-conf
RESTException [HTTP 503] [{'type': 'ERROR', 'code': None, 'text': 'KV Store initialization failed.
Please contact your system administrator.'}]

Refreshing admin/deploymentserver SplunkdConnectionException Splunkd daemon is not responding: ('Error connecting to /servicesNS/nobody/search/admin/deploymentserver/_reload: The read operation timed out',)

Refreshing admin/ingest-rfs-destinations SplunkdConnectionException Splunkd daemon is not responding: ('Error connecting to /servicesNS/nobody/search/admin/ingest-rfs-destinations/_reload: The read operation timed out',)

Refreshing admin/serverclasses SplunkdConnectionException Splunkd daemon is not responding: ('Error connecting to /servicesNS/nobody/search/admin/serverclasses/_reload: The read operation timed out',)

How we can trouble shoot this ERROR Messages ??

Re: Need a regex for these events

gcusello — Wed, 27 Sep 2023 16:10:04 GMT

Hi @Raj,

if you want to remove only a part of events, you have to follow the instructions at https://docs.splunk.com/Documentation/Splunk/9.1.1/Data/Anonymizedata

you should insert in your props.conf one SEDCMD-<class> = y/<string1>/<string2>/g, using my above regex:

SEDCMD-remove_strings = s/Data Name\=\'ParentProcessName\'\>C:\\Program Files\\(Windows Defender Advanced Threat Protection\\MsSense\.exe)|(Windows Defender Advanced Threat Protection\\SenseIR\.exe)|(AzureConnectedMachineAgent\\GCArcService\\GC\\gc_worker\.exe)|(Rapid7\\Insight Agent\\components\\insight_agent\\3\.2\.5\.31\\ir_agent\.exe)//g

Ciao.

Giuseppe