Receiving as we had to redistribute the configuration to peers and took restart i think both peers took restart when cluster master was down and now we are getting this error Cluster has only 0 peers (waiting for 2 peers to join the cluster)
Please help
Support worked with me to resolve this. The first step always check splunkd.log
Here's our WebEx recap with support team:
From the cluster master the logs made it looks like a network issue but in fact, the indexers were not starting up correctly after a cluster bundle push. After a bundle is deployed, indexers will restart. The indexers were not able to delete the $SPLUNK_HOME/slave-apps.old folder since the contents were owned by another owner.
Resolution: Manually removed the $SPLUNK_HOME/slave-apps.old folder and restarted Splunk.
Support worked with me to resolve this. The first step always check splunkd.log
Here's our WebEx recap with support team:
From the cluster master the logs made it looks like a network issue but in fact, the indexers were not starting up correctly after a cluster bundle push. After a bundle is deployed, indexers will restart. The indexers were not able to delete the $SPLUNK_HOME/slave-apps.old folder since the contents were owned by another owner.
Resolution: Manually removed the $SPLUNK_HOME/slave-apps.old folder and restarted Splunk.