Clarification on indexer retention

[mcbradford]

The documentation on this topic is not clear, so I am hoping someone can answer this for me. I need to keep data for 13 months, there are two settings within indexer.conf, frozenTimePeriodinSecs and maxTotalDataSizeMB. What is not clear is who wins? If I set maxTotalDataSizeMB to 1, but frozenTimePeriodinSeconds to 34187400, and my data is larger than 1mb, will my data roll over it reaches 1mb? Can I remove or comment out the stanza maxTotalDataSizeMB to force Splunk to just use the frozenTimePeriodinSeconds stanza?

[jkat54]

Whichever comes first wins. However, frozenTimePeriodInSecs requires the entire bucket to be older than the frozenTimePeriodInSecs. Whereas maxTotalDataSizeMB will freeze the oldest bucket(s) once it's limit is hit. This is outlined in indexes.conf documentation:

maxTotalDataSizeMB = <nonnegative integer>
* The maximum size of an index (in MB).
* If an index grows larger than the maximum size, the oldest data is frozen.
* This parameter only applies to hot, warm, and cold buckets.  It does not
  apply to thawed buckets.
* Highest legal value is 4294967295
* Defaults to 500000.

frozenTimePeriodInSecs = <nonnegative integer>
* Number of seconds after which indexed data rolls to frozen.
* If you do not specify a coldToFrozenScript, data is deleted when rolled to
  frozen.
* IMPORTANT: Every event in the DB must be older than frozenTimePeriodInSecs
  before it will roll. Then, the DB will be frozen the next time splunkd
  checks (based on rotatePeriodInSecs attribute).
* Highest legal value is 4294967295
* Defaults to 188697600 (6 years).

You cant just remove the maxTotalDataSizeMB because it will always default to 500GB.

[inventsekar · Accepted solution]

I need to keep data for 13 months, there are two settings within indexer.conf, frozenTimePeriodinSecs and maxTotalDataSizeMB.
What is not clear is who wins? If I set maxTotalDataSizeMB to 1, but frozenTimePeriodinSeconds to 34187400, and my data is larger than 1mb, will my data roll over it reaches 1mb? ///

Yes. Splunk will freeze data when either of maxtotaldatasizeMB or frozenTimePeriodInSecs is reached first.