Deployment Architecture

Max Value for bucket ID

reed_kelly
Contributor

I haven't been able to find this in the docs. The bucket ID is the last number in the name of a bucket directory. For example, 2768 is the bucket ID in the following folder name.

e.g. db_1363148387_1363078186_2768

To resolve bucket ID conflicts, we frequently add a digit to the ends of the filenames. If we do this too often, the bucket IDs could get large.

Does anyone know the maximum value that the bucket ID can have?

Tags (2)
1 Solution

ekost
Splunk Employee
Splunk Employee

On Splunk 7.0.1, *nix, x86_64, single-instance, the bucket id: db_1521417585_1518757502_999999999 was searchable at the cli, but search failed at db_1521417585_1518757502_9999999991.
The only log entry I found notable and consistent required DEBUG mode:

04–11-2018 19:20:07.650 +0000 DEBUG DatabaseDirectoryManager - Skipping path='/opt/splunk/var/lib/splunk/defaultdb/db/db_1521417585_1518757502_9999999991'

Additional errors might be noted elsewhere, but I didn't catch them.

View solution in original post

ekost
Splunk Employee
Splunk Employee

On Splunk 7.0.1, *nix, x86_64, single-instance, the bucket id: db_1521417585_1518757502_999999999 was searchable at the cli, but search failed at db_1521417585_1518757502_9999999991.
The only log entry I found notable and consistent required DEBUG mode:

04–11-2018 19:20:07.650 +0000 DEBUG DatabaseDirectoryManager - Skipping path='/opt/splunk/var/lib/splunk/defaultdb/db/db_1521417585_1518757502_9999999991'

Additional errors might be noted elsewhere, but I didn't catch them.

msearles_splunk
Splunk Employee
Splunk Employee

The bucket ID has a theoretical maximum of 2,147,483,647 (it's an int, so 2^31).

0 Karma

znaesh
Path Finder

It's an issue indeed. When we append some digits to avoid collisions, we must choose a big number (eg. 1000), but there indeed is a limit, and the bucket dirs beyond that limit are not included in .bucketManifest and not sought. And there is even no warning in logs.

0 Karma

w531t4
Path Finder

found myself asking this question earlier today -- same. i'm interested in knowing the answer.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

good question.

Get Updates on the Splunk Community!

Now Available: Cisco Talos Threat Intelligence Integrations for Splunk Security Cloud ...

At .conf24, we shared that we were in the process of integrating Cisco Talos threat intelligence into Splunk ...

Preparing your Splunk Environment for OpenSSL3

The Splunk platform will transition to OpenSSL version 3 in a future release. Actions are required to prepare ...

Easily Improve Agent Saturation with the Splunk Add-on for OpenTelemetry Collector

Agent Saturation What and Whys In application performance monitoring, saturation is defined as the total load ...