Getting Data In

Indexer fails on startup

a238574
Path Finder

When I try and restart one of my indexers after an OS upgrade I am seeing the following messages. My 2 other indexers are up and running. How do I fix this. I found one articale where they talk about fixing the offending buckets but don't say how and I am not positive this is the same issue

09-06-2018 06:37:17.576 -0400 ERROR DatabaseDirectoryManager - idx=main bid=main~392~F18EA0F4-48F1-4D8C-8209-5B40F
0B66E1E bucket=392_F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E Detected directory manually copied into its database, caus
ing id conflicts [path1='/opt/splunk/var/lib/splunk/defaultdb/db/rb_1535649215_1535592644_392_F18EA0F4-48F1-4D8C-8
209-5B40F0B66E1E' path2='/opt/splunk/var/lib/splunk/defaultdb/db/392_F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E'].
09-06-2018 06:37:17.579 -0400 ERROR IndexerService - Error intializing IndexerService: idx=main bid=main~392~F18EA
0F4-48F1-4D8C-8209-5B40F0B66E1E bucket=392_F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E Detected directory manually copied
into its database, causing id conflicts [path1='/opt/splunk/var/lib/splunk/defaultdb/db/rb_1535649215_1535592644_
392_F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E' path2='/opt/splunk/var/lib/splunk/defaultdb/db/392_F18EA0F4-48F1-4D8C-82
09-5B40F0B66E1E'].

09-06-2018 06:37:17.584 -0400 FATAL IndexerService - One or more indexes could not be initialized. Cannot disable
indexes on a clustering slave.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @a238574,

Based on logs there is Bucket ID conflict in your main index.

You have Bucket ID 392 at /opt/splunk/var/lib/splunk/defaultdb/db/rb_1535649215_1535592644_392_F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E and /opt/splunk/var/lib/splunk/defaultdb/db/392_F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E in same index which is causing issue here. The thing is bucket starting with rb_ is a replicated bucket and should replicated from different indexer but here it looks like you have same GUID - F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E across different indexers. Each Indexer should have different GUID so first check across your indexer with duplicated GUID.

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...