How to create a base search that retains multiple regex fields?

[kmaron]

I have a dashboard that is built from 3 different searches. They all come from the same data so I would like to turn them into a base search for the page. However, each one of them has a different regex to pull out a field and I can't figure out how to combine them without losing those fields.

These are my current searches:

index=foo source=bar host=PRD* CIWEB AND Error 
 | rex field=_raw "CIWEB\.(?<PluginName>.*?Plugin)"              
 | timechart span=1h count(PluginName) by PluginName

index=foo source=bar host=PRD* CIWEB AND Error 
 | rex field=_raw "\sE\_(?<ErrorType>.*?):"  
 | timechart span=1h count(ErrorType) by ErrorType

    index=foo source=bar host=PRD* CIWEB AND Error 
 | rex field=_raw "\.(?<ExceptionName>\w*?Exception)" 
 | timechart span=1h count(ExceptionName) by ExceptionName

[woodcock · Accepted solution]

Your base search is this:

index=foo source=bar host=PRD* CIWEB AND Error 
| rex "CIWEB\.(?<PluginName>.*?Plugin)"              
| rex "\sE\_(?<ErrorType>.*?):"  
| rex "\.(?<ExceptionName>\w*?Exception)" 
| multireport
   [ timechart span=1h count(PluginName)    BY PluginName    | untable _time PluginName    count]
   [ timechart span=1h count(ErrorType)     BY ErrorType     | untable _time Errortype     count]
   [ timechart span=1h count(ExceptionName) BY ExceptionName | untable _time ExceptionName count]

Then you make each post-process one of these:

fields _time PluginName count | xyseries _time PluginName count

OR:

fields _time Errortype count | xyseries _time Errortype count

OR:

fields _time ExceptionName count | xyseries _time ExceptionName count

[gemrose]

Hello,
Looking into the solution I am facing an issue when I do base-search. When I use the regex in SPL code and when it gets converted to xml. My code is not working. If I change my xml code I get unvalidated tag . Is there a way to get it working in base search

Example:

IN SPL code:

| rex field="log.mess" ".*\"Category\":\"(?<Category>[^\"]+)"

In xml:

| rex field="log.mess" ".*\"Category\":\"(?<Category>[^\"]+)"

[somesoni2]

I didn't knew of multireport command, don't see in the documentation as well. Thanks

[cmerriman]

multireport isn't in documentation. I've brought it up to the documentation team and there is a ticket with them and the engineers. it isn't fully tested out on every aspect of how it works yet.

[somesoni2]

Based on my brief testing, it runs all those timecharts (or any other aggregation command that you put in) one by one and appends the results together, making it ideal for base searches.

[cmerriman]

right. I sent @cpride_splunk my use case for it after conf last year. mine was to basically to create summation rows based on different fields. for instance:

| multireport
    [ stats count by PluginName]
    [ stats count by ErrorType PluginName]
    [ stats count by ExceptionName ErrorType PluginName]

would add summary row counts for each by statement. not necessarily for this data, but something similar to how i used it.

[kmaron]

This worked perfectly! Thank you woodcock!

[somesoni2]

Do all events have those fields that you're extracting?

[cmerriman]

you're base search could just be:

index=foo source=bar host=PRD* CIWEB AND Error 
  | rex field=_raw "CIWEB\.(?<PluginName>.*?Plugin)"       
 | rex field=_raw "\sE\_(?<ErrorType>.*?):
| rex field=_raw "\.(?<ExceptionName>\w*?Exception)" 

with each panel having a query:

 | timechart span=1h count(PluginName) by PluginName

 | timechart span=1h count(ErrorType) by ErrorType

| timechart span=1h count(ExceptionName) by ExceptionName

[kmaron]

I got that far but I thought a base search had to have a stats function? As soon as I add that it breaks.

[woodcock]

You are correct; see my answer.

[cmerriman]

try adding |table *

[somesoni2]

Or better (keep only what you need)

| table _time PluginName ErrorType ExceptionName