Why am I unable to get index discovery working with ID?

[brent_weaver]

I am using index discovery in 2 of my 3 Splunk env's and I have one that will simply not work with ID. I get the following error:

02-03-2018 13:14:23.597 +0000 ERROR IndexerDiscoveryHeartbeatThread - failed to parse response payload for group=group1, err=failed to extract FwdTarget from json node={"hostport":"?","ssl":false,"indexing_disk_space":-1}http_response=OK

I have NO IDEA why this is happening, the same automation builds this env as the others.

This is my client config:

[indexer_discovery:idx]
pass4SymmKey = MYSECRET
master_uri = https://myhost:8089

[tcpout:group1]
indexerDiscovery = idx

[tcpout]
defaultGroup = group1

Here is the master index server config:

[indexer_discovery]
pass4SymmKey = MYSECRETKEY
polling_rate = 10
indexerWeightByDiskCapacity = 0

Of course, the keys are hashed. I really hope that someone can figure this one out! I have no clue as it seems like Splunk cannot parse the file or params are missing. I can connect to the master index server from the peers on 8089 without a problem!

[brent_weaver]

Good morning, I did have one indexer that did not have the port listening and I had added the listener service (9997) and I still get the same message?!?!?

[lguinn2]

You might want to restart the master node.

[lguinn2 · Accepted solution]

I think this error message comes up when one of the indexers does not have a receiving port set. In order for indexer discovery to work, all of the indexers must have a receiving port set - the master node collects this information and then supplies it to the forwarder.

It looks like the forwarder connected to the master node correctly (your config files look fine) - but the master couldn't supply the requested information. The "hostport":"?" part of the message is what makes me believe that this is the problem.

[brent_weaver]

You my friend are an inspiration to people everywhere 🙂
THANK YOU for taking your valuable time to help me out, you were spot on!!!

[lguinn2]

(blush) You are welcome