I am setting up Distributed Search for the first time. When I enter the Peer and Authentication information, I get the red error msg;
Encountered the following error while trying to save: In handler 'distsearch-peer': Error while sending public key to search peer https://splunk11:8089: Connect timed out.
Splunk is running on the peer, but when I try to manually telnet to 8089, the connection is immediately refused. When I do a netstat -an on the box, it is not showing anything listening on 8089.
Splunk was configured to listen on a different/non-standard port. Once I solved that, I was able to get the Instances talking to each other.
Splunk indexer &
Splunk search head (ssh)
2.From indexer : Check splunk service is running : $SPLUNK_HOME/bin/splunk
Possible Error: splunkd xxxx was not running. Stopping splunk helpers... couldn't send SIGTERM to pid xxxx: Operation not permitted Couldn't send SIGTERM to some splunk helpers. [FAILED] Error: Unable to stop splunk helpers.
From indexer : Goto >> $SPLUNK_HOME/var/run/splunk/
It should work.
Possibly Firewall issues:
From indexer :
telnet localhost 8089
Check 8089 port is listening
From search head:
telnet splunk-indexer 8089
check u can able to reach 8089 port of indexer
You should have allowed in firewall rules.