Splunk Enterprise Security

No results from scheduled jobs or searches

Dharmesh_93
Loves-to-Learn Lots

Hi,

We are using Splunk cloud 8.2 and mainly utilizing for Splunk SIEM solution. 

Currently we have many scheduled alerts, searches and reports. In the recent days we could see 21% of the searches were skipped and job execution time also increased. 

From yesterday, we are unable to see output results for any of the jobs, but we are getting the search result when we execute adhoc search.

We are also able to see below Errors and warnings in our console.

The percentage of non high priority searches skipped (74%) over the last 24 hours is very high and exceeded the red thresholds (20%) on this Splunk instance. Total Searches that were part of this percentage=7056. Total skipped Searches=5271

The instance is approaching the maximum number of historical searches that can be run concurrently.

The number of extremely lagged searches (1) over the last hour exceeded the red threshold (1) on this Splunk instance

Could you please share some solution to implement in this case. 

 

 

Labels (2)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Well, there's no real magic you can do. You have to lower the load or provide more resources.

In case of resources - In case of on-premise solution it would be just about adding indexers/search-heads to your environment and/or migrating to faster storage. I'm not sure how it works with cloud but I suspect it might have something to do with license "level".

You can (and should) also review the activity on your splunk deployment - see whether you really do need all those reports, searches, alerts and so on. And see if they are properly written and if they do not unnecessarily cover too big time ranges.

0 Karma
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.


Introducing Unified TDIR with the New Enterprise Security 8.2

Read the blog
Get Updates on the Splunk Community!

.conf25 Community Recap

Hello Splunkers, And just like that, .conf25 is in the books! What an incredible few days — full of learning, ...

Splunk App Developers | .conf25 Recap & What’s Next

If you stopped by the Builder Bar at .conf25 this year, thank you! The retro tech beer garden vibes were ...

Congratulations to the 2025-2026 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...