How to find all concurrent searches that are running at the same time?

[kteng2024]

Hi there,

Is there any way to find out the all scheduled searches which are scheduled to run at same time because it seems like all the scheduled searches are running at same time causing the kernel to kill splunkd by OOM killer because splunk is consuming too much of CPU to run all those searches .

[woodcock]

You need to tune OOM Killer because Splunk need "all the stuff" routinely and that generally should not be a reason to kill it.

[skoelpin]

Try this search.. It will give you a visual representation of scheduled searches vs real-time searches

index=_internal sourcetype=splunkd source=*metrics.log group=search_concurrency "system total"
| timechart max(active_hist_searches) as "Historical Searches" min(active_realtime_searches) as "Real-time Searches" by host