topic Re: When trying to set up a distributed system, can you help me with the following error?: "Unable to distribute to peer, peer has status=2" in Getting Data In

When trying to set up a distributed system, can you help me with the following error?: "Unable to distribute to peer, peer has status=2"

xindeNokia — Tue, 23 Oct 2018 16:01:28 GMT

distributed system. splunk 7.1.2
one SH + one indexer

In the SH splunkd log:

DistributedPeerManager - Distributed: Unable to distribute to peer ..... using the uri-scheme=https because peer has status=2. Please verify uri-scheme, connectivity to the search peer, that the search peer is up, and an adequate level of system resources are available. See the Troubleshooting Manual for more information.

and it causes search failure.

what does status=2 mean? what might be happening here?

Any help is appreciated!

Re: When trying to set up a distributed system, can you help me with the following error?: "Unable to distribute to peer, peer has status=2"

xindeNokia — Fri, 02 Nov 2018 15:31:12 GMT

Just want to posted how we solved this issue in case other ppl see this issue as well - still on-going but less frequent

we suspect this is due to workload on indexer is too heavy. we dont have heavy forwarder in btw.
after we fixed couple of parsing issues on indexer. connection issue gets better.

Re: When trying to set up a distributed system, can you help me with the following error?: "Unable to distribute to peer, peer has status=2"

bgronvall_splun — Tue, 07 May 2019 19:49:36 GMT

status=2 is evaluated as "Unstable" and can only be triggered by the following two conditions.

There is a time skew between the SH and Search Peer.
The indexer is oversubscribed and rate at which it returns results is inconsistent with the other search peers.

Re: When trying to set up a distributed system, can you help me with the following error?: "Unable to distribute to peer, peer has status=2"

woodcock — Wed, 08 May 2019 00:05:04 GMT

Please do click Accept on your answer.

Re: When trying to set up a distributed system, can you help me with the following error?: "Unable to distribute to peer, peer has status=2"

cybermonday — Tue, 30 Jul 2019 15:05:40 GMT

You may want to revisit and ensure that right port used in your deployment.

Sometimes admin in config rush make mistake by sending logs to indexer on port 8089 instead of 9997 which is enough overwhelm the indexer.