topic Re: Merging similar error strings in Splunk Search

Merging similar error strings

erica — Tue, 30 Nov 2021 09:38:08 GMT

It works perfect if the difference is at the end of the strings. But I do have some additional strings that are slightly different in the middle.

My Current Query

Base search
| eval Error=message
| rex mode=sed "s/(?m)^\s+//g"
| rex field=Error mode=sed "s/^((?<Msg>.+)\s)\S+/\1*/"
| top 25 Error,file_line,level by build | table build level count file_line Error

Error String Example 1:

No exception occurred when displaying value for task=inspect entity.name=software propertyKey=keyNameForSomething. Please write a rule *

No exception occurred when displaying value for task=inspect entity.name=software propertyKey=keyNameForSomethingElse. Please write a rule *

No exception occurred when displaying value for task=inspect entity.name=software propertyKey=keyNameForSomethingElseElse. Please write a rule *

Error String Example 2

Locale is null for the language, es with ec, com.EditingContext@1y3y1u3e. Skip this *

Locale is null for the language, en with ec, com.ITEditingContext@2y5f3u3e. Skip this *

---

I would hope my output to be the following or similar:

Count,  Error

3, No exception occurred when displaying value for task=inspect entity.name=software propertyKey=*. Please write a rule *

2, Locale is null for the language, *

Re: Merging similar error strings

ITWhisperer — Tue, 30 Nov 2021 09:55:25 GMT

| rex field=Error mode=sed "s/propertyKey=[^\.]+\./propertyKey=*./g" | rex field=Error mode=sed "s/Locale is null for the language, .*/Locale is null for the language, */g"

Re: Merging similar error strings

erica — Tue, 30 Nov 2021 10:12:21 GMT

But I have a bunch of other errors, these 2 are just samples.

So I want to refrain from being too specific with the rex string

Re: Merging similar error strings

erica — Tue, 30 Nov 2021 10:13:51 GMT

I have a bunch of other error messages, so im trying to refrain from being too specific with the rex string

Re: Merging similar error strings

ITWhisperer — Tue, 30 Nov 2021 10:24:42 GMT

You don't need to be completely specific, you do however have to identify match patterns to cover each of the types of error message you wish to change. Since you only provided two types of examples, which don't appear to have a common pattern, there are two rex expressions.

Re: Merging similar error strings

PickleRick — Tue, 30 Nov 2021 11:24:15 GMT

You might want to try https://splunkbase.splunk.com/app/3109/

Disclaimer: Haven't used it myself. It's just what I found by searching for "splunk fuzzy match".

Another one is https://splunkbase.splunk.com/app/5237/

Re: Merging similar error strings

erica — Wed, 01 Dec 2021 07:23:55 GMT

ah. I was hoping there could be a pattern for those.

Thank you!

Re: Merging similar error strings

isoutamo — Wed, 01 Dec 2021 11:40:22 GMT

if you are not interested those exact error messages you can use field punct to grouping those.

... | stats values(error_msg) as error by punct ....

Probably it didn't give you a exactly what you are wanting but maybe you can use is as a starting point?

r. Ismo

Re: Merging similar error strings

erica — Wed, 01 Dec 2021 13:50:22 GMT

Hmm Im not sure how punct is used here?

it turns the error into serious of character and im not able to get any result with any grouping command
eg:

| Top 25 punct

Re: Merging similar error strings

isoutamo — Wed, 01 Dec 2021 13:56:37 GMT

https://docs.splunk.com/Splexicon:Punct

Basically punct shows somekind of pattern of _raw. This means that it also shows pattern of your error message. If those error messages are enough close to each other then those puncts are same even e.g. words are not exactly same. Maybe as I already said this is not a best option for you case but e.g. classifying errors in _internal it works quite well.