topic Re: Merging with similar strings without eval in Splunk Search

Merging with similar strings without eval

chiilii — Sat, 06 Jun 2020 23:50:19 GMT

Hi All,

I would like to combine similar strings (with different field values) in my data.

The data I have now:

Error | Count (yesterday) | Count (today)
Low ink on printer A | 10 | 0
Invalid input on line 1 | 5 | 2
Invalid input on line 2 | 4 | 4
Low ink on printer B | 6 | 3
Service crash on App1 | 1 | 0

What I want to have:

Error Type | Count (yesterday) | Count (today)
Low ink on printer * | 16 | 3
Invalid input on line * | 9 | 6
Service crash on * | 1 | 0

Note: I may have thousands of error type that needs to be combined.
Is it possible to achieve without having to eval every string?

Re: Merging with similar strings without eval

to4kawa — Thu, 30 Apr 2020 05:12:13 GMT

Error is digit + type description ?

Re: Merging with similar strings without eval

kamlesh_vaghela — Thu, 30 Apr 2020 05:12:15 GMT

@chiilii

Can you please try this ?

YOUR_SEARCH 
| rex field=Error mode=sed "s/\s(?<last>\w+)$/ * /g" max_match=0
| rex field=Error mode=sed "s/^(\d)\.\s//g" max_match=0
| stats sum("Count (yesterday)") as "Count (yesterday)" sum("Count (today)") as "Count (today)" by Error

Sample Search:

| makeresults 
| eval _raw=" Error Count (yesterday)   Count (today)
1. Low ink on printer A 10  0
2. Invalid input on line 1  5   2
3. Invalid input on line 2  4   4
4. Low ink on printer B 6   3
5. Service crash on App1    1   0" 
| multikv forceheader=1 
| rename Count__yesterday_ as "Count (yesterday)", Count__today_ as "Count (today)" 
| table Error "Count (yesterday)" "Count (today)" 
| rename comments as "this is for sample data only" 
| rex field=Error mode=sed "s/\s(?<last>\w+)$/ * /g" max_match=0
| rex field=Error mode=sed "s/^(\d)\.\s//g" max_match=0
| stats sum("Count (yesterday)") as "Count (yesterday)" sum("Count (today)") as "Count (today)" by Error

Thanks

Re: Merging with similar strings without eval

chiilii — Fri, 01 May 2020 14:47:35 GMT

@to4kawa sorry for the confusion, removed the digit

Re: Merging with similar strings without eval

chiilii — Fri, 01 May 2020 15:53:02 GMT

thanks @kamlesh_vaghela, I like how you used regex here. But what if I have a new error that has string like "Low ink on printer A and needs cartridge replacement", the outcome I'm expecting is "Low ink on printer * and needs cartridge replacement"? Would there be better way for this?

Re: Merging with similar strings without eval

to4kawa — Fri, 01 May 2020 23:37:03 GMT

| makeresults 
| eval _raw="Error,Count_yesterday,Count_today
Low ink on printer A , 10 , 0
Invalid input on line 1 , 5 , 2
Invalid input on line 2 , 4 , 4
Low ink on printer B , 6 , 3
Service crash on App1 , 1 , 0"
| rex mode=sed "s/(?m)^\s+//g" 
| multikv forceheader=1
| table E* C*
| rename COMMENT as "this is sample"

| rex field=Error mode=sed "s/^((?<Msg>.+)\s)\S+/\1*/"

| stats sum(Count_yesterday) as Count_yesterday sum(Count_today) as Count_today by Error

well, do not use eval is hard.

Re: Merging with similar strings without eval

erica — Tue, 30 Nov 2021 10:16:34 GMT

Hello!

I love your solution, but any idea how my rex string should be if my difference is in the middle of the string?

Error String Example 1:

No exception occurred when displaying value for task=inspect entity.name=software propertyKey=keyNameForSomething. Please write a rule *

No exception occurred when displaying value for task=inspect entity.name=software propertyKey=keyNameForSomethingElse. Please write a rule *

No exception occurred when displaying value for task=inspect entity.name=software propertyKey=keyNameForSomethingElseElse. Please write a rule *

Error String Example 2

Locale is null for the language, es with ec, com.EditingContext@1y3y1u3e. Skip this *

Locale is null for the language, en with ec, com.ITEditingContext@2y5f3u3e. Skip this *