Getting Data In

Trim an index down to 90 days and recover space

beaunewcomb
Communicator

So say I have an index that's got data in it back 120 Days, and I want to delete events older than 90 days, keeping the indexes trimmed to 90 days going forward. Would the below process accomplish this?

Set indexes.conf:

[indexname]
frozenTimePeriodInSecs = 7776000

restart splunk

I'm assuming that if I restart splunk, it will automatically go through and start deleting stuff older than 90 days on its own. Is this correct?

Tags (1)
0 Karma
1 Solution

emiller42
Motivator

One thing to keep in mind is that a bucket won't transition over to frozen until ALL events in the bucket exceed the timeframe given. On low volume indexes, it's possible for a bucket to have data from a wider date range, and thus would hang around longer than expected.

Another way to approach this is to estimate how much data you index in the time period desired, and then set the maxTotalDataSizeMB accordingly.

View solution in original post

emiller42
Motivator

One thing to keep in mind is that a bucket won't transition over to frozen until ALL events in the bucket exceed the timeframe given. On low volume indexes, it's possible for a bucket to have data from a wider date range, and thus would hang around longer than expected.

Another way to approach this is to estimate how much data you index in the time period desired, and then set the maxTotalDataSizeMB accordingly.

emiller42
Motivator

You can, and both will apply. So it will freeze buckets that go past the expiration time, or when the total index exceeds the size parameter. Whichever comes first.

beaunewcomb
Communicator

Can you set both the max size and time?

kristian_kolb
Ultra Champion

Yes that's right. Be VERY careful with that. Getting a few numbers wrong could let you lose a LOT of data fast 😉

0 Karma

chris
Motivator

Yes 🙂 At least thats the what happened on the index I just tried this.

Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...