Hi,
In my splunk configurtation I have defined the maxHotBuckets to default value, so 3.
When I monitor my indexers I see that the number of Hot buckets exceeds the value 3.
Can anyone explain me why the number of hot buckets can exceed the maximum number of hot buckets?
Thanks
i recommend validating from the CLI. there is not enough info from dbinspect in the search above and I am not clear on whether you are even running a cluster?? I see 1 hot per indexer in that screenshot...Jump on the cli of that indexer and confirm what you are seeing in $SPLUNK_HOME/var/lib/splunk/<yourIndex>/db
and look for the naming convention kellewic alluded to above. (hot_ and ####_GUID)
example from my standalone instance
[root@n00bserver db]# pwd
/home/splunker/splunk/var/lib/splunk/n00blab/db
[root@n00bserver db]# ls -la | grep hot
drwx--x--- 3 splunker splunker 4096 Aug 9 15:02 hot_v1_342
drwx--x--- 3 splunker splunker 4096 Aug 9 15:02 hot_v1_343
dbinspect is counting the replicated hot buckets.
Look on one of your indexers for that index; you should see 3x buckets like "hot_" and N more like "####_GUID". Those can even exceed what dbinspect says but the ones with more than just "rawdata" is what's being counted in addition to the originating hot buckets.
Or if you have multiple pipelines, this can happen as well as inventsekar pointed out.
Easy check on the replicated front; try:
|dbinspect index=INDEX
|where state="hot"
|eval replicated=if(match(path, "/rb_"), "Y", if(state="hot" AND match(path, "/\d+_[-A-F0-9]+"), "Y", "N"))
|stats count by splunk_server, replicated
|sort replicated
Those with "N" will equal 3; those with "Y" will be the remainder.
Can also check:
|rest /services/data/indexes-extended
|where title="INDEX"
|table splunk_server, maxHotBuckets, bucket_dirs.home.hot_bucket_count
https://docs.splunk.com/Documentation/Splunk/6.6.2/Admin/Indexesconf
In server.conf parallelIngestionPipelines value is 1, so the maximum number of hot bucket is always 3.
How are you monitoring them? Is it counting replicated hot buckets possibly?
I have a distributed envrironment, so I manage all my indexers on a deploployer server
I don't think it is counting replicated hot buckets.
When I use dbinspect command, I see the same result
can you be more specific and provide an example of what you are seeing?
so you are not clustering? And you are seeing more than 3 hot buckets per index? How many do you see??
I add an answer beacause I can't add img in comments