topic Re: truncating and merge values in Splunk Search

truncating and merge values

exocore123 — Thu, 29 Jun 2017 18:32:43 GMT

I have a bunch of log error descriptions that have unique IDs at the end of the sentences
"CC declined. 123" 1 "CC declined. 456" 1 "Some error. 921" 1 "Some error. 222" 1

I want to truncate the rest of the numbers and merge the count so the result looks like

"CC decined" 2 "Some error" 2

In addition, is there a way to automatically do this without stating every single instances that I want to truncate?

Re: truncating and merge values

woodcock — Thu, 29 Jun 2017 18:42:44 GMT

Show us a diverse set of raw events and the separated out description filed values.

Re: truncating and merge values

exocore123 — Thu, 29 Jun 2017 18:56:30 GMT

description         count
"CC declined. 123"     1
"CC declined. 456"     1
"Some error. 921"      1
"Some error. 222"      1
"Failed server 2"      1
"Failed server 12"     1
"CC declined. x12"     1

     description         count
"CC declined"     3
"Some error"      2
"Failed server"      2

I guess can we automatically truncate the string the moment we see any type of special character?

Re: truncating and merge values

DalJeanis — Thu, 29 Jun 2017 19:12:24 GMT

try this, where desc is the field name.

| rex field=desc "^(?<desc>(?i)[\w\s]+?)[^\w]?\s*\w?\d+$"

The complication comes from that x12 at the end of one, with no special characters in it.

Assumptions: the last character of the description field will always be a number.

Re: truncating and merge values

exocore123 — Thu, 29 Jun 2017 19:16:10 GMT

What about matching the first special character/non-word character?

Re: truncating and merge values

woodcock — Thu, 29 Jun 2017 19:24:39 GMT

Like this:

|makeresults | eval raw="CC declined. 123,1
CC declined. 456,1
Some error. 921,1
Some error. 222,1
Failed server 2,1
Failed server 12,1
CC declined. x12,1"
| makemv delim="
" raw
| mvexpand raw
| rename raw AS _raw
| rex "^(?<description>.*?),(?<count>\d+)$"
| table description count

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution."

| rex field=description mode=sed "s/\s+\d+$//"
| stats sum(count) AS count BY description

Re: truncating and merge values

exocore123 — Thu, 29 Jun 2017 19:52:19 GMT

Seems like your regex matches whitespace then number, can i terminate the moment I see a period/any special characters

Re: truncating and merge values

woodcock — Thu, 29 Jun 2017 19:53:39 GMT

It strips off all trailing digits and the white space in front of it.

Re: truncating and merge values

exocore123 — Thu, 29 Jun 2017 19:58:24 GMT

Yes, but is there a pattern to strip the moment a special character is seen, strip the rest of the string. Or strip off starting from the special character too is fine

Re: truncating and merge values

exocore123 — Thu, 29 Jun 2017 20:11:35 GMT

| rex field=description "^(?<description>[^\r\.]+)" something like this, but for many special characters

Re: truncating and merge values

exocore123 — Thu, 29 Jun 2017 20:17:39 GMT

I used | rex field=description "^(?<description>[^\r\.]+)", but I am getting "Unexpected closed tag"

Re: truncating and merge values

woodcock — Thu, 29 Jun 2017 20:50:47 GMT

Yes, like this:

| rex field=description mode=sed "s/[specialcharacterlisthere].*$//"