topic Re: Regex for field ending with $ in Splunk Dev

Regex for field ending with $

cr32003 — Thu, 15 Dec 2011 14:29:59 GMT

I am trying to filter out WMI events that have the 'User' field whicxh ends with a $ i.e a special character as far as regex is concerned.I am running Splunk version 4.2.2 on a windows 2003 server (virtualised).
my 'local' 'props.conf' is set to:-

[WMI:WinEventLog:Security]
TRANSFORMS-wmi=wmiExcludeUsers

[WMI:WinEventLog:Security]
TRANSFORMS-wmi=wmiExcludeEventCodes

and my 'local' 'transforms.conf' is set to:-

[wmiExcludeEventCodes]
REGEX=(?m)^EventCode=(512|513|514|515)
DEST_KEY=queue
FORMAT=nullQueue 

[wmiExcludeUsers]
REGEX=(?m)^User=\$$
DEST_KEY=queue
FORMAT=nullQueue

I am fairly sure that the EventCodes bit is working but the Users bit is not.

Re: Regex for field ending with $

MHibbin — Thu, 15 Dec 2011 14:42:29 GMT

Why don't you replace the "$$" with ".*\$" This should take anything up to $.

Re: Regex for field ending with $

cr32003 — Thu, 15 Dec 2011 15:13:13 GMT

Sorry the copy/paste bit obviuosly did not work correctly
the regex should read 'REGEX=(?m)^User=\$$' which i thought should be any "user value ending in '$'". I have tried changing this to what you suggested and it still has no affect.

Re: Regex for field ending with $

MHibbin — Thu, 15 Dec 2011 15:42:48 GMT

How abut something like this... I don't think you need to have two stanzas in the props.conf as they are the same sourcetype (have you checked the sourcetype is correct in Splunk?).

[WMI:WinEventLog:Security]
TRANSFORMS-wmievents = wmieventsnull 
TRANSFORMS-wmiusers = wmiusersnull

[wmieventsnull] 
REGEX = (?m)^User=(.*\$) 
DEST_KEY = queue 
FORMAT = nullQueue

[wmiexcludenull] 
REGEX = (?m)^EventCode=(512|513|514|515) 
DEST_KEY = queue 
FORMAT = nullQueue

(Thanks to Ayn for correction!)

Re: Regex for field ending with $

Ayn — Thu, 15 Dec 2011 15:52:51 GMT

The TRANSFORMS sections should have a unique identifier, so for instance TRANSFORMS-wmievents and TRANSFORMS-wmiusers.

Re: Regex for field ending with $

MHibbin — Thu, 15 Dec 2011 15:57:09 GMT

Ah ok, I wasn't sure, because you can have a transforms like...

TRANSFORMS-wmi = wmievents, wmiusers

Can't you?

Re: Regex for field ending with $

cr32003 — Mon, 19 Dec 2011 15:35:38 GMT

I have now tried pretty much all of the conbinations, including the one(s) above BUT if i put in the EventCode REGEX all is Hunky Dory and ANY changes made to the EventCode numbers are reflected in the indexing.
However if I then remove the EventCode REGEX and insert the User REGEX (either your version above or my original version) into the SAME TRANSFORMS Stanza then Splunk in its wisdom still indexes all of the events with the User field ending in '$'.
The MOST annoying thing is that if I do the following search:-
source="WMI:WinEventLog:Security" | regex User="\$$"
Then the results are all events with the User field ending in '$'.
Is there any 'debugging' tools to see what Splunk is actually doing when it receives the event and makes the decision as to whether to either index it or not?