How to measure impact of searches with long time period?

[Taruchit]

Hello All,

I need your help to understand the impact of time ranges selected by users while running their search query.

Some users may be running their SPL with longer time range such as: - All Time, 30 days, Year to date, Previous year.

Does this longer time range have any implications on Splunk search, server memory, CPU utilization, slowness or any other negative impact. And how can we measure those implications?

I reached out to seek details on the same on a different forum and got below details: -

1. Time range has massive impact on resources required.

2. It impacts speed and memory utilization.

3. It requires more resources.

Thus, I need your help to understand the impact in detail. The goal is to measure the implications of saved searches having longer time range and what is the potential gain if we identify and alter those searches with narrow and smaller time range. 

And how do we determine which time range is excessive to use among the available predefined options like:-

Last 24 hours

Last 7 days

Last 30 days

Week to date

Month to date

Year to date

and many more.

Any information from your end will be very helpful.

Thank you

[somesoni2 · Accepted solution]

To understand impact of bigger time-range, you need to understand how Splunk searching works. If you see below picture (source: Conf2016 session), especially step 3, the time range decides what and how many data buckets will be searched for filters/criteria defined in SPL. Higher the time-range, more buckets will be scanned and searched against. Most likely you'll find more matches when you search more buckets. So, the more buckets you scan and the more matches you find, your resource consumption will increase (CPU core will be help longer as search will run longer, more RAM will be used to hold search results, including intermediate results, more IO will incur to facilitate search result processing).

Which time range options will be excessive depends upon the data you're searching and how much data/buckets you've in your environment. If an index is small (both in total size and number of buckets), the impact of higher time range will be lower than when you search a larger index (larger data size and more number of buckets). You'll have to run some test searches to measure search performance for different time ranges and decide which time range is bad for your environment. Typically in larger environment, anything over 24 hours could be considered as resource intensive search.

Hope this helps.

[Taruchit]

Hi @somesoni2,

Thank you for sharing the details.

Thank you