Splunk Search

truncating and merge values

exocore123
Path Finder

I have a bunch of log error descriptions that have unique IDs at the end of the sentences

"CC declined. 123" 1
"CC declined. 456" 1
"Some error. 921" 1
"Some error. 222" 1

I want to truncate the rest of the numbers and merge the count so the result looks like

"CC decined" 2
"Some error" 2

In addition, is there a way to automatically do this without stating every single instances that I want to truncate?

0 Karma
1 Solution

woodcock
Esteemed Legend

Like this:

|makeresults | eval raw="CC declined. 123,1
CC declined. 456,1
Some error. 921,1
Some error. 222,1
Failed server 2,1
Failed server 12,1
CC declined. x12,1"
| makemv delim="
" raw
| mvexpand raw
| rename raw AS _raw
| rex "^(?<description>.*?),(?<count>\d+)$"
| table description count

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution."

| rex field=description mode=sed "s/\s+\d+$//"
| stats sum(count) AS count BY description

View solution in original post

0 Karma

woodcock
Esteemed Legend

Like this:

|makeresults | eval raw="CC declined. 123,1
CC declined. 456,1
Some error. 921,1
Some error. 222,1
Failed server 2,1
Failed server 12,1
CC declined. x12,1"
| makemv delim="
" raw
| mvexpand raw
| rename raw AS _raw
| rex "^(?<description>.*?),(?<count>\d+)$"
| table description count

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution."

| rex field=description mode=sed "s/\s+\d+$//"
| stats sum(count) AS count BY description
0 Karma

exocore123
Path Finder

Seems like your regex matches whitespace then number, can i terminate the moment I see a period/any special characters

0 Karma

woodcock
Esteemed Legend

It strips off all trailing digits and the white space in front of it.

0 Karma

exocore123
Path Finder

Yes, but is there a pattern to strip the moment a special character is seen, strip the rest of the string. Or strip off starting from the special character too is fine

0 Karma

exocore123
Path Finder

| rex field=description "^(?<description>[^\r\.]+)" something like this, but for many special characters

0 Karma

woodcock
Esteemed Legend

Yes, like this:

| rex field=description mode=sed "s/[specialcharacterlisthere].*$//"
0 Karma

DalJeanis
Legend

try this, where desc is the field name.

| rex field=desc "^(?<desc>(?i)[\w\s]+?)[^\w]?\s*\w?\d+$"

The complication comes from that x12 at the end of one, with no special characters in it.

Assumptions: the last character of the description field will always be a number.

0 Karma

exocore123
Path Finder

What about matching the first special character/non-word character?

0 Karma

exocore123
Path Finder

I used | rex field=description "^(?<description>[^\r\.]+)", but I am getting "Unexpected closed tag"

0 Karma

woodcock
Esteemed Legend

Show us a diverse set of raw events and the separated out description filed values.

0 Karma

exocore123
Path Finder
description         count
"CC declined. 123"     1
"CC declined. 456"     1
"Some error. 921"      1
"Some error. 222"      1
"Failed server 2"      1
"Failed server 12"     1
"CC declined. x12"     1

to

     description         count
"CC declined"     3
"Some error"      2
"Failed server"      2

I guess can we automatically truncate the string the moment we see any type of special character?

0 Karma
Get Updates on the Splunk Community!

Developer Spotlight with William Searle

The Splunk Guy: A Developer’s Path from Web to Cloud William is a Splunk Professional Services Consultant with ...

Major Splunk Upgrade – Prepare your Environment for Splunk 10 Now!

Attention App Developers: Test Your Apps with the Splunk 10.0 Beta and Ensure Compatibility Before the ...

Stay Connected: Your Guide to June Tech Talks, Office Hours, and Webinars!

What are Community Office Hours?Community Office Hours is an interactive 60-minute Zoom series where ...