Splunk Enterprise

Indexer cluster peers status are fluctuating?

90522prakash
Engager

Hi,

Indexer cluster peers status are fluctuating few of the times from up to pending. I have verified, there is no resources shortage problems (memory/cpu) when the indexer cluster peers fluctuated. what could be the reason for fluctuating indexer peers?
 
splunk/Health.logs are showing below messages.

 

01-02-2022 02:00:11.893 +0000 INFO PeriodicHealthReporter - feature="Indexers" color=yellow indicator="missing_peers" due_to_threshold_value=1 measured_value=3 reason="The following peers are in transition: Indexer1(Pending), Indexer2(Pending), Indexer3(Pending).
" node_type=indicator node_path=splunkd.indexer_clustering.indexers.missing_peers

 



Source watchdog/watchdog.log are showing below messages

 

01-02-2022 02:15:19.01 +0000 ERROR Watchdog - No response received from IMonitoredThread=0x7fe9c63f70 within 8000 ms. Looks like thread name='CMMasterRemoteStorageThread' tid=28158 is busy !? Starting to trace with 8000 ms interval.

01-02-2022 02:16:23.12 +0000 INFO Watchdog - Stopping trace. Response for IMonitoredThread ptr=0x7fefb70 - thread name='CMMasterRemoteStorageThread' tid=28158 - finally received after 72049 ms (estimation only).

 

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Security Highlights: September 2022 Newsletter

 September 2022 The Splunk App for Fraud Analytics (SFA) is now Splunk SupportedUse your existing Splunk ...

Platform Highlights | September 2022 Newsletter

 September 2022 What’s New in 9.0 and How to UpgradeGet a walk through of what is new Splunk Enterprise 9.0 ...

Observability Highlights | September 2022 Newsletter

 September 2022 Splunk Observability SuiteAccess to "Classic" SignalFx Interface Will be Removed on Sept 30, ...