Splunk Search

how does the attribute "maxHotIdleSecs" work?

aywong
Path Finder

In my indexes.conf file (C:\Program Files\Splunk\etc\system\local) I have the attribute "maxHotIdleSecs = 86400"

So in other words I want hot buckets to roll over to a warm bucket after it's been in existence for 24 hours.

After my initial addition of this attribute, and restart, ALL hot buckets that were over 24 hours old rolled over to warm buckets. But then the next day I had buckets that were over 24 hours old again, and they wouldn't roll over, until I restarted splunk again.

Do Indexes.conf attributes take precedence over others? Because I do have a "maxDataSize" attribute that my buckets never meet. Or should these buckets jsut roll over whenever one of these attributes are met?

Tags (2)
0 Karma
1 Solution

sowings
Splunk Employee
Splunk Employee

The keyword there is "idle". That is, if a bucket hasn't received any new indexed data for more than a day, it will get rolled to warm.

The reason that you observed a roll from hot to warm with the restart, is because Splunk does exactly that--roll from hot to warm on a restart.

If you want a bucket to contain exactly one day's worth of data, consider maxHotSpanSecs instead. Be sure to check the docs for indexes.conf!

View solution in original post

sowings
Splunk Employee
Splunk Employee

Specifically regarding the question of "Do indexes.conf attributes take precedence over others?" Splunk rolls buckets from one state to another depending upon the most restrictive of the parameters, whether that be total size, number of buckets, bucket idle time, bucket span, cumulative size of bucket state (e.g. cold, warm), cumulative size in volume group, etc.

ChrisG
Splunk Employee
Splunk Employee

Are you editing the default configuration file? You shouldn't! See About configuration files and Configuration file precedence in the Splunk Admin Manual for an explanation of how this works. You should modify $SPLUNK_HOME/etc/system/local, or $SPLUNK_HOME/etc/apps/yourappname/local.

aywong
Path Finder

I meant local, just copied the directory address I was currently in, forgot had been checking the defaults

0 Karma

jsburt
New Member

Would setting both parameters to 24 hours force each bucket to contain exactly (more or less) 24 hours worth of data for that particular index?

maxHotIdleSecs = 86400
maxHotSpanSecs = 86400

0 Karma

sowings
Splunk Employee
Splunk Employee

The keyword there is "idle". That is, if a bucket hasn't received any new indexed data for more than a day, it will get rolled to warm.

The reason that you observed a roll from hot to warm with the restart, is because Splunk does exactly that--roll from hot to warm on a restart.

If you want a bucket to contain exactly one day's worth of data, consider maxHotSpanSecs instead. Be sure to check the docs for indexes.conf!

MHibbin
Influencer

Ahh! beat me to it.. silly edits on silly IE8 (not my preference of course) 🙂

0 Karma

MHibbin
Influencer

Not too sure about this one, but I think the documentation may be misleading here:

http://docs.splunk.com/Documentation/Splunk/5.0/Admin/indexesconf

However, I would say logically, that maxHotIdleSecs would be the amount of time a hot bucket has been idle for (i.e. not being written to), not the amount of time it has been in existance. I would say maxHotSpanSecs may be better suited to your needs. Looking at the conf file spec as above. You should take note of the warning in the spec... "if you set this too small, you can get an explosion of hot/warm buckets in the filesystem", but 24 hours should be fine.

It should be noted that Splunk will roll all hot buckets when the Splunkd service is restarted, which would explain the behaviour you have observed.

Additionally...

Unless this is the default configuration, you SHOULD NOT edit the configuration files in C:\Program Files\Splunk\etc\system\default\. The recommended location for user edits would be C:\Program Files\Splunk\etc\system\local\, or at the App level (e.g. C:\Program Files\Splunk\etc\apps\<appName>\local\)

The default folder is used as the unedited location for Splunk and App developers.

aywong
Path Finder

thanks for your help 🙂

0 Karma

MHibbin
Influencer

Okay that's fine then! Hope your question has been answered!

0 Karma

aywong
Path Finder

I meant local, just copied the directory address I was currently in, forgot had been checking the defaults

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...