Monitoring Splunk

Splunkd Health Status - Search Scheduler - Search Lag & Delay

anirbandasdeb
Path Finder

Hello all, On the splunkd health report, what is the difference between Search Lag & Delay? [ref: https://docs.splunk.com/images/e/ee/Splunkd_health_report_8.0.0.png]

Our deployment has a high number of savedsearches that trigger on cron (every 5m, 15m, 30m, 1h etc) and we are working to minimise the concurrency by introducing Scheduler Window & Skew.
I know exactly which searches are triggering beyond the scheduled time (dispatch_time - scheduled_time from the scheduler.log) and which searches are skipping.
But I do not understand what Splunk signifies as Lag & Delay in terms of searches..

I have gone through the $SPLUNK_HOME/var/log/health.log & server/health/splunkd/details API endpoints but they give the same messages as the Health Indicator.. Thanks in advance!

0 Karma

janroc
Explorer

Hi,

There is an app on splunkbase named Alerts For Splunk Admins.

Have you tried the app to find the slowness?

 

Regards Jan

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...