effectivity of multiple summary searches in a single index

[bkoutsky]

Hello,

I just discovered summary indexes (Oh joy! I can have results immediately instead of waiting a few minutes) and I'm starting to use them for a lot of my searches.

The question is: should I create multiple indexes, one for each summary search, or should I put all of them into one index? How efficient is Splunk at picking up summary search events from index using 'index=summary search_name="trololo"' ? "Everything in one index" is easier to use and maintain, but I want to avoid hitting a performance wall again after I accumulate some data.

So, for a lots of data and tens of summary searches, is it significantly faster to do

index='special-index-for-search-foo'

than

index='common-index' search_name='foo'

?
cheers,
Bob

[_d_]

HI Bkoutsky,

Here is my recommendation: Yes. create multiple summary indexes but don't over do it.

I recommend creating a summary index for each search granularity. So, for example, I would have 3 summary indexes. The FIRST would be for summary searches that are less than an hour (5m, 10m, 15m, etc). I would use a SECOND one for hourly summaries up to a day and a THIRD one for summaries that are a day or longer.

I would label them like so: summary_5m, summary_1h and summary_1d

Having three summaries will allow you to set different retention policies and different rights/privileges for each, if necessary.

You can also do other cool things such as build the hourly summary index from the 5m summary and the daily summary from the hourly one!

Hope this helps!

d.

- please vote if you find this answer useful!

[AlexMcDuffMille]

Can you expand on what you mean by 'don't over do it'.

Is there a tipping point in which Splunk starts to be a lot slower? Or is it a linear curve (2 searches will be half as fast as 1)