topic Re: Remove numbers from splunk events in Splunk Search

Remove numbers from splunk events

richnavis — Fri, 21 Jun 2013 23:23:00 GMT

I"m trying to create a search that will show me the count of certain types of events I get in a Windows Event Log. The problem I'm having is that the field I'm keying on (MSG) often has unique numerical values for the same event.

Example:

Event1: There is no object at postion 2

Event2: There is no object at position 22

Event3: There is no object at position 34

Event4: Wait time has been 22 seconds

Event5: Wait time has been 45 seconds

Event6: Wait time has been 127 seconds

If I did the following search: index=eventlogs | stats count by MSG I would get 6 rows of data, each with a count of 1, where I'd really like it to tell me I have 2 events, with a count of 3 for each. In the example above, I have 2 event types, but the different unique numerical values in the data for each field makes it look like I have 6 different event types. Is there a way I can have splunk ignore the numerical values?

Re: Remove numbers from splunk events

starcher — Sat, 22 Jun 2013 00:16:28 GMT

You should should setup an eventtypes.conf. You can probably define the eventtype through the gui as well. But I am used to working in the config files.

You can do like this in eventtypes.conf

[MyEvents-NoObject]

search = index=eventlogs sourcetype=WinEventLog:Application MSG="*There is no object*"

[MyEvents-WaitTime]

search = index=eventlogs sourcetype=WinEventLog:Application MSG="*Wait time has been*"

Now with that configuration in play you should see that each of your search results has an eventtype matching it. So you could now do a search like:
index=eventlogs | stats count by eventtype

Re: Remove numbers from splunk events

gkanapathy — Sat, 22 Jun 2013 00:21:12 GMT

You can try

index=eventlogs | eval MSG=replace(MSG,"\d+","N") | stats count by MSG

Re: Remove numbers from splunk events

richnavis — Tue, 02 Jul 2013 16:13:53 GMT

Thanks.. I didn't think to use event types.. but perhaps I will use them for some other things I do. However, in this case, the number of type of events could number in the hundreds, making event types impractical.. good suggestion tho...

Re: Remove numbers from splunk events

richnavis — Tue, 02 Jul 2013 16:20:39 GMT

Nice.. is there also a way to do a similar type thing for anything between parenthesis? So.. if the event had something like "System Error(cart value=roses) has occurred", I could ignore the value inside the paraenthesis?

Re: Remove numbers from splunk events

gkanapathy — Tue, 02 Jul 2013 16:28:16 GMT

You can use any regex with replace(). So replace(MSG,"\([^\)]*\)","(P)" would work for (non-nested) parens.