topic Re: Max Value for bucket ID in Deployment Architecture

Max Value for bucket ID

reed_kelly — Wed, 27 Mar 2013 17:58:15 GMT

I haven't been able to find this in the docs. The bucket ID is the last number in the name of a bucket directory. For example, 2768 is the bucket ID in the following folder name.

e.g. db_1363148387_1363078186_2768

To resolve bucket ID conflicts, we frequently add a digit to the ends of the filenames. If we do this too often, the bucket IDs could get large.

Does anyone know the maximum value that the bucket ID can have?

Re: Max Value for bucket ID

gkanapathy — Wed, 27 Mar 2013 22:12:56 GMT

good question.

Re: Max Value for bucket ID

w531t4 — Fri, 08 Aug 2014 23:43:26 GMT

found myself asking this question earlier today -- same. i'm interested in knowing the answer.

Re: Max Value for bucket ID

znaesh — Tue, 10 Apr 2018 12:53:14 GMT

It's an issue indeed. When we append some digits to avoid collisions, we must choose a big number (eg. 1000), but there indeed is a limit, and the bucket dirs beyond that limit are not included in .bucketManifest and not sought. And there is even no warning in logs.

Re: Max Value for bucket ID

ekost — Tue, 29 Sep 2020 18:58:45 GMT

On Splunk 7.0.1, *nix, x86_64, single-instance, the bucket id: db_1521417585_1518757502_999999999 was searchable at the cli, but search failed at db_1521417585_1518757502_9999999991.
The only log entry I found notable and consistent required DEBUG mode:

04–11-2018 19:20:07.650 +0000 DEBUG DatabaseDirectoryManager - Skipping path='/opt/splunk/var/lib/splunk/defaultdb/db/db_1521417585_1518757502_9999999991'

Additional errors might be noted elsewhere, but I didn't catch them.

Re: Max Value for bucket ID

msearles_splunk — Thu, 16 Jan 2020 00:31:34 GMT

The bucket ID has a theoretical maximum of 2,147,483,647 (it's an int, so 2^31).