topic index data retention in Getting Data In

index data retention

trojan_81 — Fri, 30 Oct 2020 22:30:22 GMT

Hello there.

Within splunk cloud, I go to Settings < Indexes.

I am looking at my main index. It has a current size of 5TB and a searchable retention time of a year.

Questions:

1) How much data can remain on this index before new data starts to overwrite old data?
2) How do I view the configurations of this index such as how long the data waits before it starts going through the aging bucket stage?
3) If searchable retention is a year, does that mean data after a year will go to frozen status?

Re: index data retention

gcusello — Sat, 31 Oct 2020 07:42:34 GMT

Hi @trojan_81,

please see my ansert to you other question https://community.splunk.com/t5/Getting-Data-In/data-retention-of-sourcetype/m-p/527399#M88951

Anyway, the data remaining in index before discarding depends on the storage you bought by Splunk (in terms of dimensions and retention).

Old data are discarded when the index dimensions exceeds the maxTotalDataSizeMB value or the newest event of a bucket exceeds the retention time, retention is indipendent by how many new data you index.

In Splunk Enterprise, you can see indexes buckets status using the Monitor Console, App, probably in Splunk Cloud there's something similar but it isn't important for you, you need to know only:

max retention of your data,
max dimension of your index.

Frozen Status is only for on premise installations, when a bucket (the newest event) exceeds the retention time is discarded.

You can find more infos at https://docs.splunk.com/Documentation/SplunkCloud/8.1.2008/User/Datapolicies#:~:text=Your%20ingest%2Dbased%20Splunk%20Cloud,days%20of%20your%20uncompressed%20data.

Ciao.

Giuseppe

Re: index data retention

gcusello — Sun, 01 Nov 2020 10:06:43 GMT

Hi @trojan_81,

good for you.

Ciao and happy splunking.

Giuseppe

P.S.: Karma Points are appreciated 😉