Deployment Architecture

Max Value for bucket ID

reed_kelly
Contributor

I haven't been able to find this in the docs. The bucket ID is the last number in the name of a bucket directory. For example, 2768 is the bucket ID in the following folder name.

e.g. db_1363148387_1363078186_2768

To resolve bucket ID conflicts, we frequently add a digit to the ends of the filenames. If we do this too often, the bucket IDs could get large.

Does anyone know the maximum value that the bucket ID can have?

Tags (2)
1 Solution

ekost
Splunk Employee
Splunk Employee

On Splunk 7.0.1, *nix, x86_64, single-instance, the bucket id: db_1521417585_1518757502_999999999 was searchable at the cli, but search failed at db_1521417585_1518757502_9999999991.
The only log entry I found notable and consistent required DEBUG mode:

04–11-2018 19:20:07.650 +0000 DEBUG DatabaseDirectoryManager - Skipping path='/opt/splunk/var/lib/splunk/defaultdb/db/db_1521417585_1518757502_9999999991'

Additional errors might be noted elsewhere, but I didn't catch them.

View solution in original post

ekost
Splunk Employee
Splunk Employee

On Splunk 7.0.1, *nix, x86_64, single-instance, the bucket id: db_1521417585_1518757502_999999999 was searchable at the cli, but search failed at db_1521417585_1518757502_9999999991.
The only log entry I found notable and consistent required DEBUG mode:

04–11-2018 19:20:07.650 +0000 DEBUG DatabaseDirectoryManager - Skipping path='/opt/splunk/var/lib/splunk/defaultdb/db/db_1521417585_1518757502_9999999991'

Additional errors might be noted elsewhere, but I didn't catch them.

msearles_splunk
Splunk Employee
Splunk Employee

The bucket ID has a theoretical maximum of 2,147,483,647 (it's an int, so 2^31).

0 Karma

znaesh
Path Finder

It's an issue indeed. When we append some digits to avoid collisions, we must choose a big number (eg. 1000), but there indeed is a limit, and the bucket dirs beyond that limit are not included in .bucketManifest and not sought. And there is even no warning in logs.

0 Karma

w531t4
Path Finder

found myself asking this question earlier today -- same. i'm interested in knowing the answer.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

good question.

Get Updates on the Splunk Community!

Monitoring MariaDB and MySQL

In a previous post, we explored monitoring PostgreSQL and general best practices around which metrics to ...

Financial Services Industry Use Cases, ITSI Best Practices, and More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Splunk Federated Analytics for Amazon Security Lake

Thursday, November 21, 2024  |  11AM PT / 2PM ET Register Now Join our session to see the technical ...