I am struggling to understand data retention for Smartstore index. I guess I am thinking more from a non-smartstore way of data retention.
Below is my requirement
Data ingestion per day - 1.5 TB per day
Total retention - 90 days
Local storage - 50 days
Smartstore storage - 40 days
Multi-site indexer cluster with 2 indexers in each AWS AZ with RF - 3 and SF - 3
From the docs, I understand following parameters are key for retention
maxGlobalDataSizeMB
maxGlobalRawDataSizeMB
frozenTimePeriodInSecs
Since both maxGlobal* parameters apply to mainly warm buckets, how can I control data retention for each index as a total and not specifically to warm buckets, similar to maxTotalDataSizeMb ?
With non-smartstore index, this was so easy, you either just needed maxTotalDataSizeMb and frozenTimePeriodInSecs
Also, how can I control size of each index occupies on local storage (which is basically ability to configure something similar to homePath.maxDataSizeMB, which is again ignored for smartstore indexes)?
I am surprised all this is not documented clearly anywhere in official Splunk docs but some info can be found in forums.
How can I translate my requirements into these parameters on a per index basis ? Please help
