How to monitor data retention policy and tweak accordingly.

[davidwaugh]

I've searched but havent yet been able to find the answer.
We have a clustered index setup, and lots of data going into different indexes.

We have the indexes defined with

frozenTimePeriodInSecs

and
maxTotalDataSizeMB

I'd like to produce a dashboard if one doesnt already exist to answer the following questions:

1. What is the oldest data in each index? eg its 183 days old
2. How much of the total allotted space is each index using for its hot and cold stores.: eg 98% of Hot and 15% of Cold is being used for this index.
3. How much of the physical disks have we allocated to indexes. For example if all indexes were full, have we allocated 150% of the physical space available? All indexes sit on a HOT disk and a COLD disk. Eg 98% of hot disk is allocated, 150% of cold disk space is allocated
4. At the current rate of ingestion, what would the retention be if we used 100% of all allocated space available for the index. eg 360 days
5. What is limiting our retention - is it our maxTotalDataSizeMB or frozenTimePeriodInSecs for each index.

Thanks for your help.

Here is a screenshot showing a typical index definition that is pushed out to our index cluster.

[roseg001]

please can some one help me splunk retention policy stanza for 80 days

[gcusello · Accepted solution]

Hi davidwaugh,
I think that you should see the Monitoring Console App and check if it solves all your needs.
In addition there's the Index Usage App ( https://splunkbase.splunk.com/app/4086/ ) that could be very useful for your needs.
If there are some need that you cannot solve with them, let me know and surely you'll have the support you need.

Bye.
Giuseppe

[davidwaugh]

Thanks very much. I've just installed Index Usage and have used the Monitoring Console. I think it will take a few days to ingest the data for the dashbaords so will let you know.
Thanks for your help.

[gcusello]

Hi davidwaugh,
if you're (o when you'll be) satisfied by this answer, please accept and/or upvote it.
Bye.
Giuseppe

[davidwaugh]

Thanks Index Usage was the answer. Great app!