Monitoring Splunk

Health:Red

kmill78
Explorer

I cannot find what is doing this and the answer that google always brings back from the community is not very good. 

Root Cause(s):

  • The number of extremely lagged searches (1) over the last hour exceeded the red threshold (1) on this Splunk instance
Labels (1)
Tags (2)
0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

use the MC (monitoring console) to check what is status of scheduled searches. It should show to you which search and why is caused this warning. 
r. Ismo

kmill78
Explorer

hey thanks ! would you know where exactly in the MC ?

0 Karma

mattymo
Splunk Employee
Splunk Employee

depending on your environment, the monitoring console will be available at Settings > Monitoring Console

https://docs.splunk.com/Documentation/Splunk/8.0.5/DMC/DMCoverview

I believe search lag is a symptom of skipped or differed searches...check out the search related stats and review your server resources

 

- MattyMo

kmill78
Explorer

thank you very much ! i know how to get in to th MC , its from there i get lost 🙂 

0 Karma

mattymo
Splunk Employee
Splunk Employee

dont worry ill tie this rope to your waist....dive in, you wont get lost...docs has a guide for how to interpret the data (see link i posted above). Once you have touched the bottom of the pool come back up for some air and let us know what you working with and what you have tried!

 

mattymo_0-1595426983921.png

Otherwise, check your jobs tab in splunk and look for jobs that are queued or differed. Also it says 1 search, can you not click into it?

 

mattymo_0-1595427206401.png

 

- MattyMo

isoutamo
SplunkTrust
SplunkTrust

Hi

I think that correct item is "Scheduler Activity: Instance". If I recall right it is under the Search Head item on distributed environment?

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Why Splunk Customers Should Attend Cisco Live 2026 Las Vegas

Why Splunk Customers Should Attend Cisco Live 2026 Las Vegas     Cisco Live 2026 is almost here, and this ...

What Is the Name of the USB Key Inserted by Bob Smith? (BOTS Hint, Not the Answer)

Hello Splunkers,   So you searched, “what is the name of the usb key inserted by bob smith?”  Not gonna lie… ...

Automating Threat Operations and Threat Hunting with Recorded Future

    Automating Threat Operations and Threat Hunting with Recorded Future June 29, 2026 | Register   Is your ...