Deployment Architecture

Error moving warm buckets to new server

castle1126
Communicator

Hi, I'm in the process of taking some of my indexes and moving them to a new server. Currently both servers are up and running Splunk. I changed my forwarder to stop forwarding to the old server and send data to the new server two weeks ago.

On my old Splunk server I've manually forced a roll from hot to warm. Then I shut down both Splunk instances and SCP'd the "db_" directory from the old server to the new. When I try to bring up Splunk on the new server I get this error in the logs causing Splunk to not start:

ERROR DatabaseDirectoryManager - Splunk has detected that a directory has been manually copied into its database, causing id conflicts [/data/index1/db_1285880963_1273071936_0, hot_v1_0]

I followed a previous posting on on ANSWERS - trying to debug my issue: http://answers.splunk.com/questions/838/how-can-you-add-move-a-bucket-without-restarting-splunkd.

Tags (1)
1 Solution

hulahoop
Splunk Employee
Splunk Employee

You need to ensure the ID at the end of the bucket name is unique. This may require you rename the buckets manually. The ID is the number at the end of the warm bucket. So in your error message the bucket name is db_1285880963_1273071936_0 and the bucket ID is 0. For each Splunk index, the IDs all need to be unique.

View solution in original post

hulahoop
Splunk Employee
Splunk Employee

You need to ensure the ID at the end of the bucket name is unique. This may require you rename the buckets manually. The ID is the number at the end of the warm bucket. So in your error message the bucket name is db_1285880963_1273071936_0 and the bucket ID is 0. For each Splunk index, the IDs all need to be unique.

hulahoop
Splunk Employee
Splunk Employee

I believe only the warm and cold dbs need to have unique ids.

kevintelford
Path Finder

This worked for me as well, but I'm still not sure how the ids are now all unique. I have other indexes where in db there is a hot_v1_0 bucket and in colddb a *_0 bucket and they don't have this error. But you were right in changing it from _0 to some other number made it work.

0 Karma

castle1126
Communicator

That was the issue. I changed the bucket name to end in 500, restarted Splunk and no more errors. Thanks for the pointer!

castle1126
Communicator

Sorry for the abrupt end to my message.

Can someone give me an idea on what I did wrong.

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...