Roll hotwarm data to cold when frozenTimePeriodInSecs is met

[rickymckenzie10]

Hello, how can I ensure the data being sent to cool_index is rolled to cold when the data is 120 days old?

The config I'll use

[cool_index]
homePath = volume:hotwarm/cool_index/db
coldPath = volume:cold/cool_index/colddb
thawedPath = $SPLUNK_DB/cool_index/thaweddb
frozenTimePeriodInSecs = 10368000 #120 day retention
maxTotalDataSizeMB = 60000
maxDataSize=auto
repFactor=auto

 am I missing something?

[isoutamo · Accepted solution]

Hi

that .conf presentation which @kiran_panchavat are referring is excellent even it's little bit old and don't contains all new stuff like S2 (Splunk Smart Store). Please read it and also some other answers which are talking bout that same issue.

Shortly, You cannot ensure that events are moved into cold storage based on age! There are no parameter which define this for warm bucket. Moving warm to cold is defined base on bucket count not based on time.

frozenTimePerioInSecs is used for moving cold buckets to frozen (archiving those outside of splunk or remove those as default action).

r. Ismo

[kiran_panchavat]

@rickymckenzie10 I think that you should read at https://docs.splunk.com/Documentation/Splunk/9.2.0/Admin/Indexesconf

https://conf.splunk.com/files/2017/slides/splunk-data-life-cycle-determining-when-and-where-to-roll-... 

frozenTimePeriodInSecs = <nonnegative integer>
* The number of seconds after which indexed data rolls to frozen.
* If you do not specify a 'coldToFrozenScript', data is deleted when rolled to
frozen.
* NOTE: Every event in a bucket must be older than 'frozenTimePeriodInSecs'
seconds before the bucket rolls to frozen.
* The highest legal value is 4294967295.
* Default: 188697600 (6 years)

maxTotalDataSizeMB = <nonnegative integer>
* The maximum size of an index, in megabytes.
* If an index grows larger than the maximum size, splunkd freezes the oldest
data in the index.
* This setting applies only to hot, warm, and cold buckets. It does
not apply to thawed buckets.
* CAUTION: The 'maxTotalDataSizeMB' size limit can be reached before the time 
limit defined in 'frozenTimePeriodInSecs' due to the way bucket time spans 
are calculated. When the 'maxTotalDataSizeMB' limit is reached, the buckets 
are rolled to frozen. As the default policy for frozen data is deletion, 
unintended data loss could occur.
* Splunkd ignores this setting on remote storage enabled indexes.
* Highest legal value is 4294967295
* Default: 500000