Some of our production indexes were configured to have
frozenTimePeriodInSecs = 777600 instead of
frozenTimePeriodInSecs = 7776000, meaning, instead of 90 days they were configured to 9 days, which the client didn't like one bit when they found it.
What type of safeguards can we place to avoid such errors?
Sales Engineer is saying -
-- Configured is configured. No safeguards from a Splunk perspective.
Measure twice, cut once, have others review your changes.
Did the answer by @adonio help provide a working solution to your question? If yes, please don't forget to resolve this post by clicking "Accept". If no, please leave a comment with more feedback. Thanks!
I agree with the SE, seen it happen in the past.
double check your configurations, triple check
We have an alert that runs a few times in day to check if any index has data less than 90days and if so, we get an alert. We got bitten, when the admin deleted the buckets by mistake and had to put this alert to capture similar issues [ so, we can recover old data from backup etc..]