Received an empty peer list from the master. Waiting for peers to join the cluster. .

[msn2507]

Hi,

I am planning to set up 3 indexers and 1 search head in my Splunk envionment. As a first step I have configured 1 indexer which is my license master and 1 search head. My Indexer server.conf looks like -
[clustering]
mode = master
replication_factor = 1
search_factor = 1
pass4SymmKey = whatever

and search head server.conf looks like -
[clustering]
master_uri = https:--xx.xxx.xx.xxx:8089 (deliberately replaced // with -- asI splunk complains I dont have karma points to include include external lnks :D)
mode = slave
pass4SymmKey = whatever

after this setup when I restart Indexer and Search head, I am getting following error message in Search head -
"Received an empty peer list from the master. Waiting for peers to join the cluster. ." and all the searches are returned empty data.

Note: the management port 8089 between Search head & indexer is open.

Any help is appreciated.

[davidpaper]

See my answer to this problem here: http://answers.splunk.com/answers/104772/how-to-restart-just-a-cluster-master/114460

It doesn't eliminate the problem, but helps the Cluster Master recover faster.

[jkerai]

Couple of things to try:
1) [clustering] stanza for search head should be 'mode=searchhead'.
2) Check indexer's $SPLUNK_HOME/var/log/splunk/splunkd.log to see if there are any reported errors. Seems like it has not yet connected to master probably due to some error.
3) Access master's REST endpoint to confirm that you see indexers at https://splunk-server:8089/services/cluster/master/peers. This should list all the indexers seen by master. Seems like master does not see the indexer and thus search head also does not see them.