Splunk Search

Can someone explain concurrent historical searches to me?

sboogaar
Path Finder

I do not understand what is meant by concurrent historical searches.

Can someone else explain what it means to me?

I read the entire Splunk limits.conf, where you can specify it, but I think i missed the explanation of what a concurrent historical search actually is.

Which searches are counted? subsearches/joins?

Is it possible to view the number for a specific search?

The calculation in the docs says:

max_hist_searches = max_searches_per_cpu x number_of_cpus + base_max_searches
So does the number of cores in your CPU not matter? We have 4 searchheads with 8 cores, so by default, we can do:

1 * 4 + 6  = 10

concurrent historical searches with those machines?

For each machine we add we get only 1 concurrent historical search extra?

0 Karma

vishaltaneja070
Motivator

Hello @sboogar

A historical search is a search that has a distinct time range, such as the past hour, the previous day, or "between 2 and 4 last Tuesday". Historical searches usually review data in the past, but you can set up these searches to review events with future-dated timestamps, if your index contains them.

In Limits.conf there are different parameter which can help you out. This is the formula which is used for this:

max_searches_per_cpu x number_of_cpus + base_max_searches = max_hist_searches

The default settings are base_max_searches = 6, max_searches_per_cpu = 1, max_searches_per = 50, and max_rt_search_multiplier = These settings can be changes using limits.conf but you need to be careful as it all depends on your hardware capability.

0 Karma

sboogaar
Path Finder

@vishaltaneja07011993 Do subsearches/joins count? what if I have 5 joins in a query do I have 6 concurrent historical searches than?

0 Karma

vishaltaneja070
Motivator

No, subsearch will run first and then the base search. But there is a catch: By default, subsearches return a maximum of 10,000 results and have a maximum runtime of 60 seconds. In large production environments, it is possible that the subsearch will timeout before it completes. The best option is to rewrite the query to limit the number of events that the subsearch must process.

0 Karma

sboogaar
Path Finder

But even if all the joins return only 1 results it is counted as 6 concurrent historical searches right? so if I run 5 querys with 5 joins on the same time im doing 30 concurrent historical searches?

0 Karma

vishaltaneja070
Motivator

yes it will count as 6 searches but not concurrent as it will run one by one not at a single which can cause a block. Thats why there is a max runtime and max search result limit on subsearch.

0 Karma

sboogaar
Path Finder

@vishaltaneja07011993 I do not see the relation between blocking searches and " max runtime and max search result limit on subsearch."

0 Karma

vishaltaneja070
Motivator

@sboogaar
Don't get confused with the words, max runtime and max search result limit has been applied on subsearch so that, subsearch should not take a lot of time to run and block the main search to run.

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...