Knowledge Management
Highlighted

How to combine two accelerated saved searches to access the summary indexes of both searches?

Communicator

Hello,

I have two searches, A and B, and they are accelerated.

When I run A or B separately, job inspector shows they are accessing their specific summary indexes to get the data I need. However, when I do search A and appencols search B, it only accesses the summary index of search A, making the part of appendcols to take too long.

Is there any way to make Splunk be able to access both summary indexes of saved searches?

Thank you

0 Karma
Highlighted

Re: How to combine two accelerated saved searches to access the summary indexes of both searches?

SplunkTrust
SplunkTrust

Can you provide your search? How are you calling those accelerated saved searches (assuming) in join subsearch?

0 Karma
Highlighted

Re: How to combine two accelerated saved searches to access the summary indexes of both searches?

Esteemed Legend

Yes, show us the search.

0 Karma
Highlighted

Re: How to combine two accelerated saved searches to access the summary indexes of both searches?

Communicator
index=KK (A=1 B!="x" C!="y"  D=1) OR info OR (E=*) OR F=* OR G=* 
| timechart avg(xxx) as RT, sum(eval(if(r1=1,1,0))) as CL, sum(eval(if(isnotnull('B') OR 'AX'!="0", 1, 0))) as ERR, sum(qq) as M, sum(g) as GT, avg(eval(100*cur/max)) as T, sum(CT) as cc, avg(dp) as DD
| fields _time, CL, RT, ERR, M, GT, T, CC, D
| appendcols [ search index=KK W=* 
| bucket _time span=5m
| eval ff= mvzip('WA','WB') 
| eval fff= mvzip(ff, 'WC') 
| mvexpand fff
| rex field=fff "(?<CB>\d+),(?<CM>\d+),(?<P>.+)"
| stats sum(CB) AS cbs , sum(CM) AS cmx by _time 
| eval rate=100*(cbs/cmx)
| timechart cont=false max(rate) AS Pool%  partial=false ]
0 Karma
Highlighted

Re: How to combine two accelerated saved searches to access the summary indexes of both searches?

SplunkTrust
SplunkTrust

A nice explanation is provided by @martin_mueller in the following post where above syntax will not use the accelerated result for subsearch.

https://answers.splunk.com/answers/234242/can-searches-with-joins-be-accelerated.html#answer-234251

0 Karma
Highlighted

Re: How to combine two accelerated saved searches to access the summary indexes of both searches?

Communicator

Hi, thank you for your time.

I checked the link you posted and in the end, it shows exactly the same behaviour I am experiencing. The fact that only the first transforming command is used might be the "problem", weird though.

Thanks anyway

0 Karma
Highlighted

Re: How to combine two accelerated saved searches to access the summary indexes of both searches?

Legend

You don't actually have "summary indexes." For accelerated saved searches, Splunk automatically creates "data summaries" (also called "acceleration summaries"). These are quite different things in Splunk!

There are a few ways to do what you want. Splunk does not use an acceleration summary in a subsearch, although I can't find any documentation that says why it can't. So you might be able to trick Splunk by using the savedsearch command. I doubt that this will work, but it would be relatively easy to try.

Another way would be to actually use a summary index, and have both searches run individually to populate the summary index. Whenever you want to see the results, you then need a third search that simply retrieves the output of the two populating searches and combines them. This third search would be extremely fast. The downside is that the creation and maintenance of a summary index is a somewhat manual process - it is not automatic like report acceleration.

A final way would be to try the loadjob command. To use this technique, you would set up the second search as a scheduled search. You would need to have the scheduled search run relatively frequently, probably every 5 minutes or more frequently. Then, the overall search would include the first search and the second search (subsearch) would consist of the loadjob command. However, you would not be able to use appendcols - you would have use a join instead, to make sure that the 5-minute intervals aligned properly.

View solution in original post

Highlighted

Re: How to combine two accelerated saved searches to access the summary indexes of both searches?

Communicator

Thank you for your suggestions!

1 - The savedsearch replacing the code did not bring any better results, only the part before the subsearch used acceleration summaries.

2 - Will try that, despite the fact that automatic maintenance won't be available with that scenario.

Thanks

0 Karma
Highlighted

Re: How to combine two accelerated saved searches to access the summary indexes of both searches?

Path Finder

What about restructuring your search so it's something like this format?

index =indexA sourcetype=sourcetypeA index=indexB sourcetype=sourcetypeB | transforming commands |appendcols| putitalltogetheretc

If that's accelerated, all of the data is being pulled before the first transforming command so it should be fast.

0 Karma
Highlighted

Re: How to combine two accelerated saved searches to access the summary indexes of both searches?

Communicator

I tried that but since the subsearch uses mvzip, expand and rex, it is not worth it