Deployment Architecture

Help with Smartstore index data retention

dm1
Contributor

I am struggling to understand data retention for Smartstore index. I guess I am thinking more from a non-smartstore way of data retention.

 

Below is my requirement

Data ingestion per day - 1.5 TB per day

Total retention - 90 days

Local storage - 50 days

Smartstore storage - 40 days

Multi-site indexer cluster with 2 indexers in each AWS AZ with RF - 3 and SF - 3

From the docs, I understand following parameters are key for retention

 

 

maxGlobalDataSizeMB
maxGlobalRawDataSizeMB
frozenTimePeriodInSecs

 

 

Since both maxGlobal* parameters apply to mainly warm buckets, how can I control data retention for each index as a total and not specifically to warm buckets, similar to maxTotalDataSizeMb ?

With non-smartstore index, this was so easy, you either just needed maxTotalDataSizeMb and frozenTimePeriodInSecs

Also, how can I control size of each index occupies on local storage (which is basically ability to configure something similar to homePath.maxDataSizeMB, which is again ignored for smartstore indexes)?

 

I am surprised all this is not documented clearly anywhere in official Splunk docs but some info can be found in forums.

How can I translate my requirements into these parameters on a per index basis ? Please help

 

Tags (1)
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...