EDIT: Ignore this question.
I made the change as described, but there were a few indexes with edits in the conf file already. They happened to be the 2 biggest and the 2 I checked when I saw the freeze storm. I saw a max data life of about 90 days, correlated with the quarantine setting, and jumped to a poor conclusion.
I wish I had a better excuse than that.
Original fable:
At the recommendation from splunk support, for busy indexers, I changed the value to 7776000, or 90 days. Upon applying to my cluster, i saw a massive freeze event and lost all data older than 90d.
Working on my resume.
😞
Bogus. Situation resolved. Move along, nothing to see here.
Long story short: My mistake. This is bogus. Sorry for adding to the noise in this forum.
yeah, there must be more to this..
quarantine simply has Splunk TRY to make new buckets for any new events received whose timestamp is OLDER than 90 days ago. I say try, because depending on indexes.conf configs for hot buckets, it may have no choice but to throw it in an open bucket with the closest time to the event...
I would double check your frozenTimePeriodInSecs settings in your index's stanza as well as your global config stanza, as that is the likely culprit...not quarantine.
quarantinePastSecs = <positive integer>
* Events with timestamp of quarantinePastSecs older than "now" will be
dropped into quarantine bucket.
* This is a mechanism to prevent the main hot buckets from being polluted
with fringe events.
* Highest legal value is 4294967295
* Defaults to 77760000 (900 days).
frozenTimePeriodInSecs = <nonnegative integer>
* Number of seconds after which indexed data rolls to frozen.
* If you do not specify a coldToFrozenScript, data is deleted when rolled to
frozen.
* IMPORTANT: Every event in the DB must be older than frozenTimePeriodInSecs
before it will roll. Then, the DB will be frozen the next time splunkd
checks (based on rotatePeriodInSecs attribute).
* Highest legal value is 4294967295
* Defaults to 188697600 (6 years).
I've just set this on my home splunk, and years of data are still there. Do post your complete index config pre- and post-apply.