Knowledge Management

How does a scheduled report interval affect the summary index efficiency?

Glasses
Builder

I have a soar platform that will fetch events from a splunk summary index - which is feed by multiple scheduled searches. These scheduled searches have various schedules and the soar will fetch from the summary index via rest api of a search head. We also use the summary index for historic searching over longer periods of time, like day/weeks vs minutes or hours.

My question is whether a scheduled report with a short interval vs long interval would have a negative impact on either the accuracy or performance of a search against the summary index?

In other words, how does the interval of a scheduled search impact a summary index?

Thank you

0 Karma
1 Solution

woodcock
Esteemed Legend

The interval of the populating search determines the time-granularity and the availability of the data there. If you are running daily, you can create buckets of any size up to daily. For example, you can create 24 1-hour aggregation events but the availability will always be 1-day late(r). If you are running hourly and creating hourly aggregation events then the availability will be 1-hour late(r). Another impact is the impact of 1 search every day vs 24 searches every day. The last impact is that running hourly will consume 24X the space on disk. So:

* Data availability
* Data granularity
* Impact on disk
* Impact on infrastructure performance

View solution in original post

0 Karma

woodcock
Esteemed Legend

The interval of the populating search determines the time-granularity and the availability of the data there. If you are running daily, you can create buckets of any size up to daily. For example, you can create 24 1-hour aggregation events but the availability will always be 1-day late(r). If you are running hourly and creating hourly aggregation events then the availability will be 1-hour late(r). Another impact is the impact of 1 search every day vs 24 searches every day. The last impact is that running hourly will consume 24X the space on disk. So:

* Data availability
* Data granularity
* Impact on disk
* Impact on infrastructure performance
0 Karma
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...