Deployment Architecture

Max Value for bucket ID

reed_kelly
Contributor

I haven't been able to find this in the docs. The bucket ID is the last number in the name of a bucket directory. For example, 2768 is the bucket ID in the following folder name.

e.g. db_1363148387_1363078186_2768

To resolve bucket ID conflicts, we frequently add a digit to the ends of the filenames. If we do this too often, the bucket IDs could get large.

Does anyone know the maximum value that the bucket ID can have?

Tags (2)
1 Solution

ekost
Splunk Employee
Splunk Employee

On Splunk 7.0.1, *nix, x86_64, single-instance, the bucket id: db_1521417585_1518757502_999999999 was searchable at the cli, but search failed at db_1521417585_1518757502_9999999991.
The only log entry I found notable and consistent required DEBUG mode:

04–11-2018 19:20:07.650 +0000 DEBUG DatabaseDirectoryManager - Skipping path='/opt/splunk/var/lib/splunk/defaultdb/db/db_1521417585_1518757502_9999999991'

Additional errors might be noted elsewhere, but I didn't catch them.

View solution in original post

ekost
Splunk Employee
Splunk Employee

On Splunk 7.0.1, *nix, x86_64, single-instance, the bucket id: db_1521417585_1518757502_999999999 was searchable at the cli, but search failed at db_1521417585_1518757502_9999999991.
The only log entry I found notable and consistent required DEBUG mode:

04–11-2018 19:20:07.650 +0000 DEBUG DatabaseDirectoryManager - Skipping path='/opt/splunk/var/lib/splunk/defaultdb/db/db_1521417585_1518757502_9999999991'

Additional errors might be noted elsewhere, but I didn't catch them.

msearles_splunk
Splunk Employee
Splunk Employee

The bucket ID has a theoretical maximum of 2,147,483,647 (it's an int, so 2^31).

0 Karma

znaesh
Path Finder

It's an issue indeed. When we append some digits to avoid collisions, we must choose a big number (eg. 1000), but there indeed is a limit, and the bucket dirs beyond that limit are not included in .bucketManifest and not sought. And there is even no warning in logs.

0 Karma

w531t4
Path Finder

found myself asking this question earlier today -- same. i'm interested in knowing the answer.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

good question.

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...