Reporting

What are the differences between the various features named "summary" in Splunk?

mataharry
Communicator

They are many features using objects named "summary", this is confusing, please clarify.

what are the differences between all those paths ?

$SPLUNK_HOME/var/lib/splunk/summary/db
$SPLUNK_HOME/var/lib/splunk/defaultdb/summary 
$SPLUNK_HOME/var/lib/splunk/defaultdb/datamodel_summary

In savedsearches, what means auto_summarize and alert.action=summary

1 Solution

yannK
Splunk Employee
Splunk Employee

To clarify there are 3 features named "summary" in splunk, :

A - Summary indexing : classic since splunk 4.*

  • populated by scheduled searches, can use special "si*" stats commands (use the parameter alert.action=summary in savedsearches.conf)
  • results are saved in the spooler and reindexed with the sourcetype stash_new
  • stored in an index of your choice.
  • an index named "summary" is shipped with splunk by default ($SPLUNK_HOME/var/lib/splunk/summary/db)
  • the results have to be retrieved with special searches syntax
  • docs : http://docs.splunk.com/Documentation/Splunk/6.1.3/Knowledge/Usesummaryindexing

B - Report acceleration : introduced on splunk 5.*

C - Data model acceleration : introduced on splunk 6.*

Remark : none of those features counts on your license usage, but they can add some extra search load to generate the summarized data.

View solution in original post

yannK
Splunk Employee
Splunk Employee

To clarify there are 3 features named "summary" in splunk, :

A - Summary indexing : classic since splunk 4.*

  • populated by scheduled searches, can use special "si*" stats commands (use the parameter alert.action=summary in savedsearches.conf)
  • results are saved in the spooler and reindexed with the sourcetype stash_new
  • stored in an index of your choice.
  • an index named "summary" is shipped with splunk by default ($SPLUNK_HOME/var/lib/splunk/summary/db)
  • the results have to be retrieved with special searches syntax
  • docs : http://docs.splunk.com/Documentation/Splunk/6.1.3/Knowledge/Usesummaryindexing

B - Report acceleration : introduced on splunk 5.*

C - Data model acceleration : introduced on splunk 6.*

Remark : none of those features counts on your license usage, but they can add some extra search load to generate the summarized data.

jaredlaney
Contributor

@yannK - Is there anyway you could explain these in more of a conceptual vs. a mechanical way?

0 Karma

jaredlaney
Contributor

For example, maybe explain if one is more like an additional index vs. one being a cache? Maybe some good cases of when to use one vs. when to use another.

0 Karma

yannK
Splunk Employee
Splunk Employee

Those methods A and B are not supposed to complete each others they are just 2 ways to achieve the same thing.

A - The "Summary indexing" is like generating events in a new index.
It's is perfect to generate a new set of pre-calculated data, and keep it for a longer retention.
example : having millions of web acccess logs in an index with a short retention, and every day summarize them as a number of hit per day, store in an dedicated index with a long retention. At the end you will only keep this information.
The only difficulty is if a scheduled search is skipped, you may have a gap to backfill

B - Report acceleration is for searches only, it precaculate them for you.
Example : having a long statistical search over a long period to populate a dashboard. Accelerate it to run all the time in the background , and load faster.

C - Data model acceleration is only usefull if you already have a datamodel. They are usually heavier to run, so accelerating them will help.
Example : the Common Information Model (CIM) comes with many datamodels, once the volume is large, the searches are slower. When the acceleration is turned on (depending of thebackfill range), it will be faster for the recent days.

mic
Splunk Employee
Splunk Employee

yannK mentioned the following

an index named "summary" is shipped with splunk by default ($SPLUNK_HOME/var/lib/splunk/summary/db)

I believe it's $SPLUNK_HOME/var/lib/splunk/summarydb, not $SPLUNK_HOME/var/lib/splunk/summary/db. Notice that there is no backslash between "summary" and "db".

0 Karma
Get Updates on the Splunk Community!

Maximize the Value from Microsoft Defender with Splunk

<P style=" text-align: center; "><span class="lia-inline-image-display-wrapper lia-image-align-center" ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

<FONT size="5"><FONT size="5" color="#FF00FF">Get the latest news and updates from the Splunk Community ...