Monitoring Splunk

How to get to know peer down time with SPL?

brandy81
Path Finder

Hi Guru! (I edited)

I have indexer cluster and one search head. I do not use monitoring console. One of peer nodes has been shutdown and the server as well. It seems that the indexer has been shut down due to OS issue.

How am I able to get to know the exact shutdonw time using SPL? I would be index=_internal.... . Could you please help me out?

Labels (2)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @brandy81,
I don't know if this solution could answer to your need:

| metasearch index=_internal 
| timechart span=10m count BY host
| where count=0
| delta _time AS diff
| where diff=600
| stats sum(diff) AS total

In this way you have the sum (in seconds) of all the periods without internal logs, that means Splunk not active.
You can also calculate a percentage with the analysis period (e.g. 24 hours).

Ciao.
Giuseppe

venkateshparank
Path Finder

Can you try below query ?

index=_internal source="/opt/splunk/var/log/splunk/splunkd.log" component=CMPeer peer_name="ADD_YOUR_PEER_NAME_HERE" to=Stopped

OR

index=_internal component=CMPeer peer_name="ADD_YOUR_PEER_NAME_HERE" stop*

0 Karma

brandy81
Path Finder

@venkateshparankusam Hi, Thank you sooooo much for your response. The instances are not able to access remotely, so please let me check within a couple of days. I will let you know the result and accept your answer. Hope it would work. Thank you again.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Observability Simplified: Combining User Experience, Application Performance & ...

Tech Talk Observability Simplified: Combining User Experience, Application Performance & Network ...

Event Series May & June: From Network Visibility to Service Intelligence

Unifying the Network: Moving from Alert Noise to Service Intelligence with Splunk ITSI In today’s hybrid ...

Global Splunk User Group Events: May + June 2026

Your Splunk Community Awaits: Discover Upcoming User Group Events Worldwide    Staying ahead in the fast-paced ...