Deployment Architecture

Received an empty peer list from the master. Waiting for peers to join the cluster. .

msn2507
Path Finder

Hi,

I am planning to set up 3 indexers and 1 search head in my Splunk envionment. As a first step I have configured 1 indexer which is my license master and 1 search head. My Indexer server.conf looks like -
[clustering]
mode = master
replication_factor = 1
search_factor = 1
pass4SymmKey = whatever

and search head server.conf looks like -
[clustering]
master_uri = https:--xx.xxx.xx.xxx:8089 (deliberately replaced // with -- asI splunk complains I dont have karma points to include include external lnks :D)
mode = slave
pass4SymmKey = whatever

after this setup when I restart Indexer and Search head, I am getting following error message in Search head -
"Received an empty peer list from the master. Waiting for peers to join the cluster. ." and all the searches are returned empty data.

Note: the management port 8089 between Search head & indexer is open.

Any help is appreciated.

Tags (1)
0 Karma

davidpaper
Contributor

See my answer to this problem here: http://answers.splunk.com/answers/104772/how-to-restart-just-a-cluster-master/114460

It doesn't eliminate the problem, but helps the Cluster Master recover faster.

0 Karma

jkerai
Splunk Employee
Splunk Employee

Couple of things to try:
1) [clustering] stanza for search head should be 'mode=searchhead'.
2) Check indexer's $SPLUNK_HOME/var/log/splunk/splunkd.log to see if there are any reported errors. Seems like it has not yet connected to master probably due to some error.
3) Access master's REST endpoint to confirm that you see indexers at https://splunk-server:8089/services/cluster/master/peers. This should list all the indexers seen by master. Seems like master does not see the indexer and thus search head also does not see them.

Get Updates on the Splunk Community!

Splunk Cloud | Empowering Splunk Administrators with Admin Config Service (ACS)

Greetings, Splunk Cloud Admins and Splunk enthusiasts! The Admin Configuration Service (ACS) team is excited ...

Tech Talk | One Log to Rule Them All

One log to rule them all: how you can centralize your troubleshooting with Splunk logs We know how important ...

Splunk Security Content for Threat Detection & Response, Q1 Roundup

Join Principal Threat Researcher, Michael Haag, as he walks through: An introduction to the Splunk Threat ...