Getting Data In

Help filter out unwanted data from indexing using nullqueue Please

marcusmartin
Path Finder

Hi if someone could please help that would be great, I have events showing up in the indexer that are pushing me over my license, alot of it is useless to me information and i have been trying to wrap my head around filtering it out using regex but i just cant get my head around it. 

Below is a typical event i would like to rid my indexer of, i cant just block all the events with 4634  as some of them are valid, but i would like to block all events where the "Targetusersid" is similar to DOMAIN\ABC-12345$  

Can anyone help

<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{54849625-5478-4994-A5BA-3E3B0328C30D}'/><EventID>4634</EventID><Version>0</Version><Level>0</Level><Task>12545</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime='2021-02-12T08:24:29.977950700Z'/><EventRecordID>314243098</EventRecordID><Correlation/><Execution ProcessID='852' ThreadID='12388'/><Channel>Security</Channel><Computer>domaincontoller.domainname</Computer><Security/></System><EventData><Data Name='TargetUserSid'>DomainName\machine-name$</Data><Data Name='TargetUserName'>Machine-Name$</Data><Data Name='TargetDomainName'>DomainName</Data><Data Name='TargetLogonId'>0x22b9251d</Data><Data Name='LogonType'>3</Data></EventData></Event>

Props.conf 

[XmlWinEventLog:Security]
TRANSFORMS-xml = xmlnull
REGEX=(?m)^EventCode="(4662|566)" Message="Object Type:(?!\s*groupPolicyContainer)"
REGEX=(?m)^EventCode="(4656|4670|4663|4703|4658|4688)" Message="Account Name:(\W+\w+$)"
REGEX=(?m)^EventCode="(4688|4689)" Message="%SplunkUniversalForwarder%"
REGEX=(?m)^EventCode="6278" Message="Network Policy Server granted full access to a user because the host met the defined health policy."
REGEX=(?m)^EventCode="(4624|4634|4627|4648)" Message="Account\sName:.*[\S\s]*Account\sName:\s+[\S+]+[\$]"

Transforms.conf

[xmlnull]
REGEX= NO idea 😞
DEST_KEY = queue
FORMAT = nullQueue

 

Labels (2)
0 Karma
1 Solution

scelikok
SplunkTrust
SplunkTrust

Hi @marcusmartin,

Since you are using XML format you can filter events using $XmlRegex directive. There is no need to use nullQueue. Also the REGEX settings in props does not work, you should put them into inputs.conf on universal forwarder changing to XML format.

You can use below for filtering 

inputs.conf

[WinEventLog://Security]
blacklist2 = $XmlRegex="<EventID>4634<\/EventID>.*<Data Name='TargetUserSid'>[^\\]+\\[^<]+\$"

Please try below;

Please also look at below document for filtering unnecessary data from Windows logs. You can use this filters also.

https://docs.splunk.com/Documentation/WindowsAddOn/8.1.1/User/Configuration#Configure_event_cleanup_... 

If this reply helps you an upvote is appreciated.

View solution in original post

scelikok
SplunkTrust
SplunkTrust

Hi @marcusmartin,

You can try below; (You should escape "\")

blacklist2 = $XmlRegex="<EventID>4634<\/EventID>.*<Data Name='TargetUserSid'>NT AUTHORITY\\SYSTEM"

 

If this reply helps you an upvote is appreciated.

marcusmartin
Path Finder

blacklist5 = $XmlRegex="NT AUTHORITY\\SYSTEM"

 

I used this in the end as i really wasnt bothered what event id it came from im not intrested in events from that particular system account. Seems to be working and yes you are right i just learned today that you have to escape any \ and put \\ for the regex to handle it.

thanks again for your time its very much appreciated.

Cuppa coffee to you. 

 

Regards

 

0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @marcusmartin,

Since you are using XML format you can filter events using $XmlRegex directive. There is no need to use nullQueue. Also the REGEX settings in props does not work, you should put them into inputs.conf on universal forwarder changing to XML format.

You can use below for filtering 

inputs.conf

[WinEventLog://Security]
blacklist2 = $XmlRegex="<EventID>4634<\/EventID>.*<Data Name='TargetUserSid'>[^\\]+\\[^<]+\$"

Please try below;

Please also look at below document for filtering unnecessary data from Windows logs. You can use this filters also.

https://docs.splunk.com/Documentation/WindowsAddOn/8.1.1/User/Configuration#Configure_event_cleanup_... 

If this reply helps you an upvote is appreciated.

marcusmartin
Path Finder

Could I be a real pain How would you block TargetUserSid'>NT AUTHORITY\SYSTEM using regex?

0 Karma

marcusmartin
Path Finder

blacklist2 = $XmlRegex="<EventID>4634<\/EventID>.*<Data Name='TargetUserSid'>NT AUTHORITY\SYSTEM"

 

I just tried this but im thick when it comes to regex

0 Karma

marcusmartin
Path Finder

Perfect response, thankyou so much i would never have figured it out im sure. much respect.

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...