Deployment Architecture

Re-enabling a disabled cluster peer throwinf erros

nivedita_viswan
Path Finder

I have 2 peer nodes in my cluster - 1 of them was acting as a stand-alone indexer while the other is a fresh installation.
I had enabled clustering on the stand-alone indexer initially, but due to some errors, I disabled it.
I know have clustering enabled on both the peers and they are both recognized by the master node.

However, I now see the following messages - Search peer has the following message: Too many streaming errors to target=. Not rolling hot buckets on further errors to this target. (This condition might exist with
other targets too. Please check the logs.)

I understand re-enabling clustering on a disabled peer node is a known issue. I read this document - http://docs.splunk.com/Documentation/Splunk/6.1.3/Indexer/Bucketreplicationissues
The suggestion is to clean the hot buckets or remove the standalone buckets on the peer before re-enabling it. There dont seem to be any instructions on how to do this.
Also, will I lose an indexed data by doing this? As I mentioned, the node was acting as a stand-alone indexer and was indexing important data before enabling clustering.

0 Karma

svasan_splunk
Splunk Employee
Splunk Employee

nivedita_viswanath,

That error might mean a network issue (or more precisely a consistent replication issue ) between the two nodes.

See http://docs.splunk.com/Documentation/Splunk/6.1.3/Indexer/Bucketreplicationissues#Network_issues_imp...

Is the replication port configured properly on the two nodes. Are there any errors related to that in the splunkd logs?

0 Karma

nivedita_viswan
Path Finder

Can someone please help me out with this question?

0 Karma
Get Updates on the Splunk Community!

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...

Modern way of developing distributed application using OTel

Recently, I had the opportunity to work on a complex microservice using Spring boot and Quarkus to develop a ...