Splunk not respecting maxHotSpanSecs of 1day

[goncalocoelho]

Hi all,

I have specific situation where I need to roll buckets form hot to warm on a daily basis, for an index with very low volume of aprox. 20-30MB each day.

The problem is that the config maxHotSpanSecs = 86400 is not being respected, and but the buckets remain in hot state. I tried also several combinations with maxHotBuckets=2 or auto, but nothing changes.

Can someone help? When does Splunk decide to create new hot buckets?

Is it possible that the input volume is so low (in comparison with the default maxDataSize of 750MB) that Splunk won't enforce the maxHotSpanSecs of 1 day?

[thetech]

G'day,

Hot buckets are created when data is ingested into Splunk. Hot buckets are rolled to warm when one of the following occurs.

• "MaxDataSize" limit is reached - The maximum size, in megabytes, that a hot bucket can reach before splunkd
  triggers a roll to warm.
• Lifetime of the hotbucket is older than "maxHotSpanSecs"
• Manual rollover of Hot bucket
• Indexer restart

With MaxDataSize set to auto (750MB) and maxHotSpanSecs set to 86400(1 Day) they should roll over as you require (assuming the bucket size is ~20-30MB).

Are you able to send through what you have configured in you indexes.conf (Both global and per index).

*EDIT*

I missed your title statement "maxHotSpanSecs of 1h" - You need to roll every hour? maxHotSpanSecs would need to be configured as 3600 (1 Hour).  Please note 3600 is the minimum setting for maxHotSpanSecs

Either way let me know/send through your current config

Regards

theTech

[goncalocoelho]

Hi @thetech thanks for your reply!

(I misspelled the title, its is supposed to be "...maxHotSpanSecs of 1day". Sorry about that...)

Yes I'm aware of the several ways in which the buckets roll from hot to warm. And that is what's puzzling me...

My current config is this. (I have a cluster of 2 Indexers)

[my_index]
coldPath = volume:primary/my_index/colddb
homePath = volume:primary/my_index/db
thawedPath = $SPLUNK_DB/my_index/thaweddb
maxHotSpanSecs = 86400
maxHotIdleSecs = 86400

When I run a | dbinspect I get the following results, which indicates that there is one bucket (on each IDX) that still hasn't rolled