Settings for 1year retention period

[jkamdar]

I have this small Splunk Enterprise deployment in a lab that's air gapped.

So I setup this deployment about 18 months ago. Recently I noticed, I am not rolling any data. I want to set retention period of 1 year for all the data. After checking the configuration, looks like I have # of Hot buckets set to auto (which is 3 by default, I assume) but I don't find any Warm buckets. So, everything is in Hot buckets. I am looking at few settings maxHotSpanSecs, frozenTimePeriodInSecs and maxVolumeDataSizeMB, that should roll data to warm and then cold buckets eventually. 

Under /opt/splunk/etc/system/local/indexes.conf
maxHotSpanSecs is set to 7776000
frozenTimePeriodInSecs 31536000
maxVolumeDataSizeMB (not set)

Under /opt/splunk/etc/apps/search/indexs.conf
maxHotSpanSecs not set
frozenTimePeriodInSecs 31536000 (for all the indexes)
maxVolumeDataSizeMB (not set)

Shouldn't frozenTimePeriodInSecs take precedent?

Maybe, my maxVolumeDataSizeMB is set to too high. Do I need to change it? How do frozenTimePeriodInSecs and maxVolumeDataSizeMB affect each other? I thought frozenTimePeriodInSecs would override maxVolumeDataSizeMB

[jkamdar · Accepted solution]

Ok, I have set maxHotSpanSecs 86400. I am seeing lots of warm buckets now. With and frozenTimePeriodInSecs 31536000, I think, I am seeing results that I was hoping for.

Also, with the below search, I am seeing data being rolled to frozen as well.

index=_internal sourcetype=splunkd log_level=INFO component=BucketMover "freeze succeeded"

[richgalloway]

You say you found no warm buckets, but what about cold buckets?  Did you find any of those?

If you're not running out of disk space then maxVolumeDataSizeMB is not too high.

The current settings have buckets spanning 90 days.  Therefore, you should have 5 "generations" of buckets - 0-90 days (hot), 91-180 days, 181-270 days, 271-360 days, and 361-450 days.  That last one is because a bucket won't be frozen until *all* events in it exceed the retention time.

Set maxHotSpanSecs to 86400 so each bucket only contains a single day of data and retention should improve.

[livehybrid]

Hi @jkamdar 

Due to having things defined in different places here, it might be best to run a btool to the exact configuration (including default values)

Please could you run:

$SPLUNK_HOME/bin/splunk cmd btool indexes list --debug <yourIndexName>

When you talk about buckets "not rolling" - do you mean from Hot->Warm, or Cold->Frozen?

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[jkamdar]

Thanks @livehybrid and @richgalloway both suggestions are helpful. I was able to use btool to find what indexes.conf each index is using and then I did change  maxHotSpanSecs to the suggested # and I see more warm buckets. If this going to trigger data deletion that's over an year old, that's great, I will wait and see.

However, regardless what was set for maxHotSpanSecs, shouldn't frozenTimePeriodInSecs have triggered the expiration of data and delete? O

I sure am not clear how maxHotSpanSecs and frozenTimePeriodInSecs work together and affects the retention period. If one can explain, it would be great. 

[richgalloway]

The frozenTimePeriodInSecs setting does not apply to hot buckets.  You should, however, see warm buckets once a hot bucket fills up or becomes 90 days old.  I can't explain why you don't.