Deployment Architecture

Help with Smartstore index data retention


I am struggling to understand data retention for Smartstore index. I guess I am thinking more from a non-smartstore way of data retention.


Below is my requirement

Data ingestion per day - 1.5 TB per day

Total retention - 90 days

Local storage - 50 days

Smartstore storage - 40 days

Multi-site indexer cluster with 2 indexers in each AWS AZ with RF - 3 and SF - 3

From the docs, I understand following parameters are key for retention






Since both maxGlobal* parameters apply to mainly warm buckets, how can I control data retention for each index as a total and not specifically to warm buckets, similar to maxTotalDataSizeMb ?

With non-smartstore index, this was so easy, you either just needed maxTotalDataSizeMb and frozenTimePeriodInSecs

Also, how can I control size of each index occupies on local storage (which is basically ability to configure something similar to homePath.maxDataSizeMB, which is again ignored for smartstore indexes)?


I am surprised all this is not documented clearly anywhere in official Splunk docs but some info can be found in forums.

How can I translate my requirements into these parameters on a per index basis ? Please help


Tags (1)
0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Updates (ESCU) - New Releases

In the last month, the Splunk Threat Research Team (STRT) has had 3 releases of new content via the Enterprise ...

Thought Leaders are Validating Your Hard Work and Training Rigor

As a Splunk enthusiast and member of the Splunk Community, you are one of thousands who recognize the value of ...

.conf23 Registration is Now Open!

Time to toss the .conf-etti 🎉 —  .conf23 registration is open!   Join us in Las Vegas July 17-20 for ...