topic Re: how to take out numeric numbers at the end of the event? in Splunk Search

how to take out numeric numbers at the end of the event?

axl88 — Thu, 20 Mar 2014 20:30:50 GMT

Hi all,

I need little help from good Regexp guy, or may be i m so bad that the guy could be moderate.
I have a log file that i need to categorize errors.

my regexp is : (?i)^(?:[^ ]* ){8}(?P<MY_VALUE>.+)(?=\d{8}\.) -> matches first line, ignores any other type

here is the 2 events:
Mask failed to grab activity for response 1234567.
Error in Application for PID 3131: System.Exception: Error processing the following address:

So my question is I want to match both lines. Problem there is, in order to identify overall performance for my system, I need to get rid of ID numbers.(it is at end of the first line, ignore the digits after PID- I need them 🙂 ) So first line should match until numbers whereas second line matched totally.

I tried some but I couldn't figure out how to ensure that.

Thanks in advance for your time and effort.

Re: how to take out numeric numbers at the end of the event?

somesoni2 — Thu, 20 Mar 2014 21:00:01 GMT

You want to remove them from the event data itself or you want to remove them from just the search result?

Re: how to take out numeric numbers at the end of the event?

axl88 — Thu, 20 Mar 2014 21:18:02 GMT

It is search result. I can't remove event data as I ll be getting it remotely.
I have two panels in dashboard,
User selection in "master view" would detail chosen value in "detail view".
I m on the step to detail logs for detail view.

Since JavaScript would come to play, I should create generic extracted field for each log that I can minimize the complexity in JavaScript.

My plan is to extract field with the pattern I desire above
and use it for a search that will come through JS perma-link.

Re: how to take out numeric numbers at the end of the event?

somesoni2 — Thu, 20 Mar 2014 21:26:12 GMT

And the numbers at the end, they follow some pattern? like have dot "." at the end or always are 7-8 digits?

Re: how to take out numeric numbers at the end of the event?

axl88 — Thu, 20 Mar 2014 21:52:05 GMT

I have both patterns unfortunately.
Some ends with ".", some with just number.
But these are the only possibilities with the number at the end of the line.
All digits in the events are 8 digits.
When you ask for it, I just realized that we may possibly get rid of all 8 digit numbers, right.

Re: how to take out numeric numbers at the end of the event?

martin_mueller — Thu, 20 Mar 2014 23:53:00 GMT

To get rid of a number at the end of an event at search time you can do this:

base search | rex mode=sed "s/\s*\d+(\.\d+)\s*$//"

That looks for a number, possibly with decimal places, and removes it along with surrounding whitespace if and only if the number is at the end of the event. You can restrict this to a specific field by specifying field=name.

Note, your example event had an additional literal dot at the end of the event - if that's accurate you'll need to amend the regex like this:

s/\s*\d+(\.\d+)\s*\.?$//

Re: how to take out numeric numbers at the end of the event?

axl88 — Fri, 21 Mar 2014 18:08:48 GMT

please read my question again with the comments.

Re: how to take out numeric numbers at the end of the event?

martin_mueller — Fri, 21 Mar 2014 21:02:21 GMT

I did. Now what?