I have Splunk 8.0.0 on AWS with a clustered indexer set up (1 Master and 4 indexers) and I have deployed custom test apps (with basic monitoring for windows/Linux logs) on the servers that have the forwarders installed. I have enabled the indexer discovery feature in the outputs.conf file (local folder) for these apps and on the server.conf file of the cluster master (etc/system/local) but I see the following error in the forwarder logs:
04-05-2020 16:57:53.752 +1000 ERROR IndexerDiscoveryHeartbeatThread - Error in Indexer Discovery communication. Verify that the pass4SymmKey set under [indexer_discovery:target1] in 'outputs.conf' matches the same setting under [indexer_discovery] in 'server.conf' on the Cluster Master. [uri=https://clustermaster:8089/services/indexer_discovery http_code=502 http_response="Error connecting: Connect Timeout"]
I have ensured that the pass4SymmKey attribute is the same for the outputs.conf on the forwarders and the server.conf on the cluster master (in their respective indexer discovery sections), but yet I see this error.
Any pointers that would help me resolve this?
... View more