Help filter out unwanted data from indexing using nullqueue Please

[marcusmartin]

Hi if someone could please help that would be great, I have events showing up in the indexer that are pushing me over my license, alot of it is useless to me information and i have been trying to wrap my head around filtering it out using regex but i just cant get my head around it. 

Below is a typical event i would like to rid my indexer of, i cant just block all the events with 4634  as some of them are valid, but i would like to block all events where the "Targetusersid" is similar to DOMAIN\ABC-12345$  

Can anyone help

<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{54849625-5478-4994-A5BA-3E3B0328C30D}'/><EventID>4634</EventID><Version>0</Version><Level>0</Level><Task>12545</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime='2021-02-12T08:24:29.977950700Z'/><EventRecordID>314243098</EventRecordID><Correlation/><Execution ProcessID='852' ThreadID='12388'/><Channel>Security</Channel><Computer>domaincontoller.domainname</Computer><Security/></System><EventData><Data Name='TargetUserSid'>DomainName\machine-name$</Data><Data Name='TargetUserName'>Machine-Name$</Data><Data Name='TargetDomainName'>DomainName</Data><Data Name='TargetLogonId'>0x22b9251d</Data><Data Name='LogonType'>3</Data></EventData></Event>

Props.conf 

[XmlWinEventLog:Security]
TRANSFORMS-xml = xmlnull
REGEX=(?m)^EventCode="(4662|566)" Message="Object Type:(?!\s*groupPolicyContainer)"
REGEX=(?m)^EventCode="(4656|4670|4663|4703|4658|4688)" Message="Account Name:(\W+\w+$)"
REGEX=(?m)^EventCode="(4688|4689)" Message="%SplunkUniversalForwarder%"
REGEX=(?m)^EventCode="6278" Message="Network Policy Server granted full access to a user because the host met the defined health policy."
REGEX=(?m)^EventCode="(4624|4634|4627|4648)" Message="Account\sName:.*[\S\s]*Account\sName:\s+[\S+]+[\$]"

Transforms.conf

[xmlnull]
REGEX= NO idea 😞
DEST_KEY = queue
FORMAT = nullQueue

[scelikok]

Hi @marcusmartin,

You can try below; (You should escape "\")

blacklist2 = $XmlRegex="<EventID>4634<\/EventID>.*<Data Name='TargetUserSid'>NT AUTHORITY\\SYSTEM"

[marcusmartin]

blacklist5 = $XmlRegex="NT AUTHORITY\\SYSTEM"

I used this in the end as i really wasnt bothered what event id it came from im not intrested in events from that particular system account. Seems to be working and yes you are right i just learned today that you have to escape any \ and put \\ for the regex to handle it.

thanks again for your time its very much appreciated.

Cuppa coffee to you. 

Regards

[scelikok · Accepted solution]

Hi @marcusmartin,

Since you are using XML format you can filter events using $XmlRegex directive. There is no need to use nullQueue. Also the REGEX settings in props does not work, you should put them into inputs.conf on universal forwarder changing to XML format.

You can use below for filtering 

inputs.conf

[WinEventLog://Security]
blacklist2 = $XmlRegex="<EventID>4634<\/EventID>.*<Data Name='TargetUserSid'>[^\\]+\\[^<]+\$"

Please try below;

Please also look at below document for filtering unnecessary data from Windows logs. You can use this filters also.

https://docs.splunk.com/Documentation/WindowsAddOn/8.1.1/User/Configuration#Configure_event_cleanup_... 

[marcusmartin]

Could I be a real pain How would you block TargetUserSid'>NT AUTHORITY\SYSTEM using regex?

[marcusmartin]

blacklist2 = $XmlRegex="<EventID>4634<\/EventID>.*<Data Name='TargetUserSid'>NT AUTHORITY\SYSTEM"

I just tried this but im thick when it comes to regex

[marcusmartin]

Perfect response, thankyou so much i would never have figured it out im sure. much respect.