Deployment Architecture
Highlighted

Why does the number of temporary buckets exceed the number of replicated buckets in an indexer cluster?

Path Finder

Hi All,

I am following this document to get an understanding about the indexer cluster bucket replication in Splunk.
http://docs.splunk.com/Documentation/Splunk/6.0/Indexer/HowSplunkstoresindexes#Warm.2Fcold_bucket_na...

Excerpt
Note: In an indexer cluster, when data is streaming from the originating peer to a target peer, the data first goes into a temporary directory on the target peer, which is identified by the originating peer's &ltlocalid&gt and &ltguid&gt, like this: &ltlocalid&gt&ltguid&gt. This is true, independent of the type of bucket the data is being streamed from. When the replication has completed, the directory is rolled into a warm bucket, identified by the rb prefix, as described above.

Also parallely I had a demo indexer cluster system for my testing. Interestingly during my tests I found that the temporary buckets (localidguid) are in larger number than the actual **rbbuckets** (replicated buckets).

What could be the possible explanation for this ?

Any help appreciated thanks!

0 Karma
Highlighted

Re: Why does the number of temporary buckets exceed the number of replicated buckets in an indexer cluster?

Path Finder

Is there a specific property which controls the size of the temporary bucket before it renames to replicated bucket?

0 Karma
Highlighted

Re: Why does the number of temporary buckets exceed the number of replicated buckets in an indexer cluster?

Splunk Employee
Splunk Employee

The temporary buckets you describe are searchable hot buckets. They're just actively open for writing "now". You'll wait for a few possibilities: the bucket going idle (maxHotIdleSecs), the bucket being crowded out (maxHotBuckets), reaching its largest size (maxDataSize, default "auto" or 750MB unless your index is "main", in which case it's "autohighvolume" which is 10 GB on a 64-bit platform). Also, a restart (on the "start" part of the stop/start) will force buckets to roll from hot to warm. There are a few edge cases in an indexing cluster related to a loss of connectivity with the cluster master as well.

In any event, just wait, and eventually you'll have many more rb than the hot searchable copies!

View solution in original post