Strict Time Retention Policy

[hiph151]

Hi,

we want to implement a strict 120 day time retention policy for some indexes.
So this config should be fine. For my understanding it´s mandatory to set als MaxHotspanSecs for 24h (of course the frozen path)

MaxHotSpanSecs=86400 (1day)
FrozenTimePeriodinSec= 10368000 (120 days)

Thank you

[hiph151]

Thx for your answers! I participated on the Splunk Admin course and in the learning PDF there is a simple example:
HR data has to be frozen after 90 days, but not sooner. Also there is a hint: If a bucket spans more than one day, you cant meet the 90 day requirement.

I have a real project now, and our requerements are 120 days so im thinking about these maxHotSpanSecs to reach our 120 days. but if you say, the "frozenTimePeriodinSecs" is enough then im also statisfied 🙂

thx

[richgalloway]

Setting `MaxHotSpanSecs' to a day or less is a good idea in this case. It helps to ensure each bucket only contains a single day of data so you can better enforce your retention policy.

[koshyk]

MaxHotSpanSecs is NOT mandatory as it will go to the default value if not set. Defaults to 7776000 seconds (90 days)

All the setting you need is (example)

[_introspection]
homePath   = volume:home/_introspection/db
coldPath = volume:cold/_introspection/colddb
thawedPath = volume:cold/_introspection/thaweddb
tstatsHomePath = volume:tstats/_introspection/datamodel_summary
# Let volumes handle size, set high limit per index (set to100GB*120 days). Just to be safe
maxTotalDataSizeMB = 12288000
# 120 days total retention
frozenTimePeriodInSecs = 10368000
repFactor = auto

Another good thing is to test your indexes.conf in a Development system by putting frozentime to 2 days etc. Just to see if all works well. Also good practice to set "indexer volume" accordingly.

[richgalloway]

That setting for FrozenTimePeriodinSec should do the job. It may not be as strict as you expect, however. The setting is not enforced until the newest event in a bucket is past the time specified so there could be events much older than 120 days by the time the bucket is deleted. It depends on how many events fit into the bucket and how active the index is.

Why do you think MaxHotSpanSecs must be set to one day? The default is 90 days so, obviously, it can have other values.