Splunk Search

Can I see the top skipped searches?

danielbb
Motivator

Is there a way to categorize the skipped searches by volume, by time of invocation, etc? We are trying to understand which searches are being skipped and why?

Tags (2)
0 Karma

gjanders
SplunkTrust
SplunkTrust

The monitoring console works well as per the above posts, alternatively in Alerts for Splunk Admins (SplunkBase), the simplified version of:
AllSplunkEnterpriseLevel - Splunk Scheduler skipped searches and the reason (github)

Is:

index=_internal sourcetype=scheduler status=skipped source=*scheduler.log  
| fillnull concurrency_category concurrency_context concurrency_limit
| stats count, earliest(_time) AS firstSeen, latest(_time) AS lastSeen by savedsearch_id, reason, app, concurrency_category, concurrency_context, concurrency_limit, search_type, user, host 
| eval firstSeen = strftime(firstSeen, "%+"), lastSeen=strftime(lastSeen, "%+")

Please hit accept on the most appropriate answer, although up voting is also appreciated 🙂

richgalloway
SplunkTrust
SplunkTrust

The Monitoring Console (Settings->Monitoring Console->Search->Scheduler Activity) offers several breakdowns of skipped searches over time. You can click the magnifying glass icon in any of them to open the panel in Search so you can customize it as desired.

---
If this reply helps you, Karma would be appreciated.

danielbb
Motivator

Mostly it's about - The maximum number of concurrent running jobs for this historical scheduled search on this instance has been reached (2).

0 Karma

richgalloway
SplunkTrust
SplunkTrust

If you don't find that in in the MC try this query.

index=_internal source=*scheduler.log "The maximum number of concurrent running jobs for this historical scheduled search" 
| timechart count by savedsearch_name
---
If this reply helps you, Karma would be appreciated.
0 Karma

aberkow
Builder

Hey - check out the answer I posted here - https://answers.splunk.com/answers/790088/splunk-searches-delayed.html#answer-790351, I leverage the Monitoring Console really heavily for their built in searches!

Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...