How to prevent extremely old data from cooling / freezing

[jocobknight]

Hello,
I'm using Enron emails as test data for a training project, and I'm setting the timestamp to match the sent date of each email.

The problem, though, is that the Enron emails roughly span between 1996 and 2005, which means they are between 23 and 14 years old, as far as Splunk is concerned after indexing.

How would I configure my indexes.conf stanza to retain data in the hot buckets indefinitely? I've tried it twice and Splunk is deleting my data. I'm using the default settings for a new index. Here is the current stanza:

[enron]
coldPath = $SPLUNK_DB\enron\colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB\enron\db
maxTotalDataSizeMB = 10240
thawedPath = $SPLUNK_DB\enron\thaweddb

Thank you!

[richgalloway · Accepted solution]

Add frozenTimePeriodInSecs = 788400000 to your indexes.conf stanza to save the data for 25 years.

[jocobknight]

Thanks @richgalloway!

I was in the middle of building a modified coldToFrozen script that would instantly thaw frozen data because of the whole "If Your Only Tool Is a Hammer Then Every Problem Looks Like a Nail" mentality.

But in my heart I knew there had to be some simple solution with conf edits. I couldn't find anything on a specific retention time frame for old & static data sets. (It's all about intake rate and calculated bucket fill-ups, etc.)

But I digress, thanks again!

[jocobknight]

Just wanted to say it's been over a week and the data still hasn't dropped. Thanks again!

[jocobknight]

Alternately, does anyone know how I could take hot data and move it directly to thawed? I see in Splunk docs that the thawed data does not suffer from retention policy enforcement.