how does the attribute "maxHotIdleSecs" work?

[aywong]

In my indexes.conf file (C:\Program Files\Splunk\etc\system\local) I have the attribute "maxHotIdleSecs = 86400"

So in other words I want hot buckets to roll over to a warm bucket after it's been in existence for 24 hours.

After my initial addition of this attribute, and restart, ALL hot buckets that were over 24 hours old rolled over to warm buckets. But then the next day I had buckets that were over 24 hours old again, and they wouldn't roll over, until I restarted splunk again.

Do Indexes.conf attributes take precedence over others? Because I do have a "maxDataSize" attribute that my buckets never meet. Or should these buckets jsut roll over whenever one of these attributes are met?

[sowings]

Specifically regarding the question of "Do indexes.conf attributes take precedence over others?" Splunk rolls buckets from one state to another depending upon the most restrictive of the parameters, whether that be total size, number of buckets, bucket idle time, bucket span, cumulative size of bucket state (e.g. cold, warm), cumulative size in volume group, etc.

[ChrisG]

Are you editing the default configuration file? You shouldn't! See About configuration files and Configuration file precedence in the Splunk Admin Manual for an explanation of how this works. You should modify $SPLUNK_HOME/etc/system/local, or $SPLUNK_HOME/etc/apps/yourappname/local.

[aywong]

I meant local, just copied the directory address I was currently in, forgot had been checking the defaults

[jsburt]

Would setting both parameters to 24 hours force each bucket to contain exactly (more or less) 24 hours worth of data for that particular index?

maxHotIdleSecs = 86400
maxHotSpanSecs = 86400

[sowings · Accepted solution]

The keyword there is "idle". That is, if a bucket hasn't received any new indexed data for more than a day, it will get rolled to warm.

The reason that you observed a roll from hot to warm with the restart, is because Splunk does exactly that--roll from hot to warm on a restart.

If you want a bucket to contain exactly one day's worth of data, consider maxHotSpanSecs instead. Be sure to check the docs for indexes.conf!

[MHibbin]

Ahh! beat me to it.. silly edits on silly IE8 (not my preference of course) 🙂

[MHibbin]

Not too sure about this one, but I think the documentation may be misleading here:

http://docs.splunk.com/Documentation/Splunk/5.0/Admin/indexesconf

However, I would say logically, that maxHotIdleSecs would be the amount of time a hot bucket has been idle for (i.e. not being written to), not the amount of time it has been in existance. I would say maxHotSpanSecs may be better suited to your needs. Looking at the conf file spec as above. You should take note of the warning in the spec... "if you set this too small, you can get an explosion of hot/warm buckets in the filesystem", but 24 hours should be fine.

It should be noted that Splunk will roll all hot buckets when the Splunkd service is restarted, which would explain the behaviour you have observed.

Additionally...

Unless this is the default configuration, you SHOULD NOT edit the configuration files in C:\Program Files\Splunk\etc\system\default\. The recommended location for user edits would be C:\Program Files\Splunk\etc\system\local\, or at the App level (e.g. C:\Program Files\Splunk\etc\apps\<appName>\local\)

The default folder is used as the unedited location for Splunk and App developers.

[aywong]

thanks for your help 🙂

[MHibbin]

Okay that's fine then! Hope your question has been answered!

[aywong]

I meant local, just copied the directory address I was currently in, forgot had been checking the defaults