Health Monitor Engine - Scheduler: Lags and Skipped

[verbal_666]

Hi there.
I would like to know about Splunk Health engine, Enterprise 8.2.12, 3 SHC,

1. HOW it considers a savedsearch a Lagged search? Based on same previous 24h search runs and doing an average running times? Since we have many many heavy searches that end up also in 10/15m
2. WHY, sometimes, i found in Skipped search monitor a 100% of skipped search (1 from 1, when we have hundreds of scheduled searches)? WHILE, searching the scheduler log, i found something like 70.000 success / 68 skipped (scheduled every minute or every two, concurrency is a factor i calculate and there's no problem) in last 24h ? WHY 100%? Is it a bug? I also search for a single scheduled search per day savedsearches, but all (few) are in "success" status 🙄

When those strange things occur, sometimes, restarting the cluster, make health monitor to reset without warnings!!! Other times, in reverse, restarting the cluster make a clean health monitor to start giving warnings from point 1 & 2 🙄 ... strange behaviour!!! 😒

Thanks.

[isoutamo]

Hi

this could explain that behaviour to you https://docs.splunk.com/Documentation/Splunk/9.1.2/DMC/Configurefeaturemonitoring

Based on this instructions you could see what those health messages means.

r. Ismo