topic Re: How to create a base search that retains multiple regex fields? in Splunk Search

How to create a base search that retains multiple regex fields?

kmaron — Mon, 22 May 2017 20:00:23 GMT

I have a dashboard that is built from 3 different searches. They all come from the same data so I would like to turn them into a base search for the page. However, each one of them has a different regex to pull out a field and I can't figure out how to combine them without losing those fields.

These are my current searches:

index=foo source=bar host=PRD* CIWEB AND Error 
 | rex field=_raw "CIWEB\.(?<PluginName>.*?Plugin)"              
 | timechart span=1h count(PluginName) by PluginName

index=foo source=bar host=PRD* CIWEB AND Error 
 | rex field=_raw "\sE\_(?<ErrorType>.*?):"  
 | timechart span=1h count(ErrorType) by ErrorType

    index=foo source=bar host=PRD* CIWEB AND Error 
 | rex field=_raw "\.(?<ExceptionName>\w*?Exception)" 
 | timechart span=1h count(ExceptionName) by ExceptionName

Re: How to create a base search that retains multiple regex fields?

cmerriman — Mon, 22 May 2017 20:07:43 GMT

you're base search could just be:

index=foo source=bar host=PRD* CIWEB AND Error 
  | rex field=_raw "CIWEB\.(?<PluginName>.*?Plugin)"       
 | rex field=_raw "\sE\_(?<ErrorType>.*?):
| rex field=_raw "\.(?<ExceptionName>\w*?Exception)"

with each panel having a query:

 | timechart span=1h count(PluginName) by PluginName

 | timechart span=1h count(ErrorType) by ErrorType

| timechart span=1h count(ExceptionName) by ExceptionName

Re: How to create a base search that retains multiple regex fields?

kmaron — Mon, 22 May 2017 20:09:44 GMT

I got that far but I thought a base search had to have a stats function? As soon as I add that it breaks.

Re: How to create a base search that retains multiple regex fields?

cmerriman — Mon, 22 May 2017 20:17:22 GMT

try adding |table *

Re: How to create a base search that retains multiple regex fields?

somesoni2 — Mon, 22 May 2017 22:02:38 GMT

Do all events have those fields that you're extracting?

Re: How to create a base search that retains multiple regex fields?

somesoni2 — Mon, 22 May 2017 22:05:44 GMT

Or better (keep only what you need)

| table _time PluginName ErrorType ExceptionName

Re: How to create a base search that retains multiple regex fields?

woodcock — Mon, 22 May 2017 23:09:55 GMT

Your base search is this:

index=foo source=bar host=PRD* CIWEB AND Error 
| rex "CIWEB\.(?<PluginName>.*?Plugin)"              
| rex "\sE\_(?<ErrorType>.*?):"  
| rex "\.(?<ExceptionName>\w*?Exception)" 
| multireport
   [ timechart span=1h count(PluginName)    BY PluginName    | untable _time PluginName    count]
   [ timechart span=1h count(ErrorType)     BY ErrorType     | untable _time Errortype     count]
   [ timechart span=1h count(ExceptionName) BY ExceptionName | untable _time ExceptionName count]

Then you make each post-process one of these:

fields _time PluginName count | xyseries _time PluginName count

OR:

fields _time Errortype count | xyseries _time Errortype count

OR:

fields _time ExceptionName count | xyseries _time ExceptionName count

Re: How to create a base search that retains multiple regex fields?

woodcock — Tue, 23 May 2017 00:18:50 GMT

You are correct; see my answer.

Re: How to create a base search that retains multiple regex fields?

kmaron — Tue, 23 May 2017 15:21:43 GMT

This worked perfectly! Thank you woodcock!

Re: How to create a base search that retains multiple regex fields?

somesoni2 — Tue, 23 May 2017 15:39:50 GMT

I didn't knew of multireport command, don't see in the documentation as well. Thanks

Re: How to create a base search that retains multiple regex fields?

cmerriman — Tue, 23 May 2017 15:47:57 GMT

multireport isn't in documentation. I've brought it up to the documentation team and there is a ticket with them and the engineers. it isn't fully tested out on every aspect of how it works yet.

Re: How to create a base search that retains multiple regex fields?

somesoni2 — Tue, 23 May 2017 15:58:28 GMT

Based on my brief testing, it runs all those timecharts (or any other aggregation command that you put in) one by one and appends the results together, making it ideal for base searches.

Re: How to create a base search that retains multiple regex fields?

cmerriman — Tue, 23 May 2017 16:06:44 GMT

right. I sent @cpride_splunk my use case for it after conf last year. mine was to basically to create summation rows based on different fields. for instance:

| multireport
    [ stats count by PluginName]
    [ stats count by ErrorType PluginName]
    [ stats count by ExceptionName ErrorType PluginName]

would add summary row counts for each by statement. not necessarily for this data, but something similar to how i used it.

Re: How to create a base search that retains multiple regex fields?

gemrose — Fri, 22 Mar 2024 14:03:22 GMT

Hello,
Looking into the solution I am facing an issue when I do base-search. When I use the regex in SPL code and when it gets converted to xml. My code is not working. If I change my xml code I get unvalidated tag . Is there a way to get it working in base search

Example:

IN SPL code:

| rex field="log.mess" ".*\"Category\":\"(?<Category>[^\"]+)"

In xml:

| rex field="log.mess" ".*\"Category\":\"(?&lt;Category&gt;[^\"]+)"