Deployment Architecture

How do i set a retention period for any index...?

Builder

I am bit confused on setting up a retention period for an Index in splunk....If i need a retention period of 60 days setting a frozenTimePeriodInSecs = 5184000 will work, if i go with the settings below,
How does the rolling happen from hot to warm, warm to cold and thawed...?
Does this setting says the data is searchable for 30day period before it's deleted...?
What' the typical way to setting this with other options...?

[jra_server]
homePath = <path>
coldPath = <path>
thawedPath = <path>
frozenTimePeriodInSecs = 5184000
Labels (1)
1 Solution

Ultra Champion

Since 60 * 60* 24 * 60 = 5184000 you are all set. Data should be searchable for 60 days before it's being deleted.
The best practice is to set frozenTimePeriodInSecs only and not interfere with the internal buckets set-up.

View solution in original post

Esteemed Legend

Unless you have a requirement to purge (i.e. you MUST NOT keep data any longer than X), I would handle it by size only. Just allocate PLENTY of space to hold your data (maybe 2X) and then keep an eye on it. If you add in other settings, then it gets trickier to monitor and you do not have as many monitoring options. You also have NO BUFFER when you are getting close to deleting because of space: all of a sudden you will be losing data and it is gone. If you just work with size, you can see (set an alert) to track how often data is being archived and you can set a threshold to notify you when you retention SPACE is so small that it is only holding 65 days worth of data. Now you have early warning and time to get more space BEFORE data is lost.

Builder

So, it's good to go with maxTotalDataSizeMB instead going with forzenTimePeriodInSecs..?

0 Karma

Esteemed Legend

Yes, exactly. What is the point in dropping it just because you "officially shouldn't need it", especially when you have the space to store "a little bit extra, a little but longer, just in case"?

0 Karma

Builder

got it, Thank you..!!

0 Karma

Ultra Champion

Since 60 * 60* 24 * 60 = 5184000 you are all set. Data should be searchable for 60 days before it's being deleted.
The best practice is to set frozenTimePeriodInSecs only and not interfere with the internal buckets set-up.

View solution in original post

Builder

Thanks,...what's the advantage of defining these kind of attribute-values in indexes.conf...?

maxMemMB = 20
maxConcurrentOptimizes = 6
maxHotIdleSecs = 86400
maxHotBuckets = 10

0 Karma

Explorer

Here is way to list all configuration via btool.
splunk btool indexes list --debug

0 Karma

Ultra Champion

You know, there are many config parameters, which we can look at one by one, but as long as all works fine, sticking to the defaults is a good practice.

0 Karma

Builder

Thanks ddrillic..:)

0 Karma

Builder

Thank you make sense..

0 Karma