Splunk Search

How to create a base search that retains multiple regex fields?

kmaron
Motivator

I have a dashboard that is built from 3 different searches. They all come from the same data so I would like to turn them into a base search for the page. However, each one of them has a different regex to pull out a field and I can't figure out how to combine them without losing those fields.

These are my current searches:

index=foo source=bar host=PRD* CIWEB AND Error 
 | rex field=_raw "CIWEB\.(?<PluginName>.*?Plugin)"              
 | timechart span=1h count(PluginName) by PluginName

index=foo source=bar host=PRD* CIWEB AND Error 
 | rex field=_raw "\sE\_(?<ErrorType>.*?):"  
 | timechart span=1h count(ErrorType) by ErrorType

    index=foo source=bar host=PRD* CIWEB AND Error 
 | rex field=_raw "\.(?<ExceptionName>\w*?Exception)" 
 | timechart span=1h count(ExceptionName) by ExceptionName
1 Solution

woodcock
Esteemed Legend

Your base search is this:

index=foo source=bar host=PRD* CIWEB AND Error 
| rex "CIWEB\.(?<PluginName>.*?Plugin)"              
| rex "\sE\_(?<ErrorType>.*?):"  
| rex "\.(?<ExceptionName>\w*?Exception)" 
| multireport
   [ timechart span=1h count(PluginName)    BY PluginName    | untable _time PluginName    count]
   [ timechart span=1h count(ErrorType)     BY ErrorType     | untable _time Errortype     count]
   [ timechart span=1h count(ExceptionName) BY ExceptionName | untable _time ExceptionName count]

Then you make each post-process one of these:

fields _time PluginName count | xyseries _time PluginName count

OR:

fields _time Errortype count | xyseries _time Errortype count

OR:

fields _time ExceptionName count | xyseries _time ExceptionName count

View solution in original post

woodcock
Esteemed Legend

Your base search is this:

index=foo source=bar host=PRD* CIWEB AND Error 
| rex "CIWEB\.(?<PluginName>.*?Plugin)"              
| rex "\sE\_(?<ErrorType>.*?):"  
| rex "\.(?<ExceptionName>\w*?Exception)" 
| multireport
   [ timechart span=1h count(PluginName)    BY PluginName    | untable _time PluginName    count]
   [ timechart span=1h count(ErrorType)     BY ErrorType     | untable _time Errortype     count]
   [ timechart span=1h count(ExceptionName) BY ExceptionName | untable _time ExceptionName count]

Then you make each post-process one of these:

fields _time PluginName count | xyseries _time PluginName count

OR:

fields _time Errortype count | xyseries _time Errortype count

OR:

fields _time ExceptionName count | xyseries _time ExceptionName count

View solution in original post

somesoni2
Revered Legend

I didn't knew of multireport command, don't see in the documentation as well. Thanks

0 Karma

cmerriman
Super Champion

multireport isn't in documentation. I've brought it up to the documentation team and there is a ticket with them and the engineers. it isn't fully tested out on every aspect of how it works yet.

0 Karma

somesoni2
Revered Legend

Based on my brief testing, it runs all those timecharts (or any other aggregation command that you put in) one by one and appends the results together, making it ideal for base searches.

0 Karma

cmerriman
Super Champion

right. I sent @cpride_splunk my use case for it after conf last year. mine was to basically to create summation rows based on different fields. for instance:

| multireport
    [ stats count by PluginName]
    [ stats count by ErrorType PluginName]
    [ stats count by ExceptionName ErrorType PluginName]

would add summary row counts for each by statement. not necessarily for this data, but something similar to how i used it.

0 Karma

kmaron
Motivator

This worked perfectly! Thank you woodcock!

0 Karma

somesoni2
Revered Legend

Do all events have those fields that you're extracting?

0 Karma

cmerriman
Super Champion

you're base search could just be:

index=foo source=bar host=PRD* CIWEB AND Error 
  | rex field=_raw "CIWEB\.(?<PluginName>.*?Plugin)"       
 | rex field=_raw "\sE\_(?<ErrorType>.*?):
| rex field=_raw "\.(?<ExceptionName>\w*?Exception)" 

with each panel having a query:

 | timechart span=1h count(PluginName) by PluginName

 | timechart span=1h count(ErrorType) by ErrorType

| timechart span=1h count(ExceptionName) by ExceptionName

kmaron
Motivator

I got that far but I thought a base search had to have a stats function? As soon as I add that it breaks.

0 Karma

woodcock
Esteemed Legend

You are correct; see my answer.

0 Karma

cmerriman
Super Champion

try adding |table *

0 Karma

somesoni2
Revered Legend

Or better (keep only what you need)

| table _time PluginName ErrorType ExceptionName
0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!