We are trying to troubleshoot some memory consumption issues with one of the SH cluster nodes.
We found that this instance shows high concurrency of scheduled reports 46/15 historical while the other nodes are way below this number.
Also in the running historical scheduled reports panel we got a column Mode that shows "historical" as value.
What does a "historical" report mean in this context?
The Splunk documentation for DMC doesnt explain it.
https://docs.splunk.com/Documentation/Splunk/9.0.3/DMC/Scheduleractivity
Regards.
https://docs.splunk.com/Splexicon:Historicalsearch
https://docs.splunk.com/Splexicon:Historicalsearch
Thanks for the reply. Very helpful, but i still cant clearly understand what the number 46 means.
Are the 46 concurrent historical searches that shows this panel scheduled and running at this moment?
Or are they currently scheduled to run at another time (not running)?
The left number (46) is highest number of searches running at any time in the past (not sure how far back it looks). The right number (15) is the configured maximum number of concurrent searches.
I guess we got something wrong with the SH cluster, it is not scheduling the searches evenly across the nodes.
And the one with the highest count is not even the captain.
Thanks again for the reply, we ll open a support case and try to troubleshoot this issue.