Splunk Search

how to set the retention policy of index

jichen
Explorer

Hi,I'm also confusing about the retention policy. I want to keep some indexes for 90 days. Now I'm doing some test,when I set maxHotIdleSecs no more than 180,it works fine, it'll archive the index about per 5 minutes. But when I set to 200, 300,600 .... ,it doesn't archive the data according the time I set or doesn't delete the retired data at all.

[os]
coldPath = $SPLUNK_DB\os\colddb
homePath = $SPLUNK_DB\os\db
thawedPath = $SPLUNK_DB\os\thaweddb
coldToFrozenDir = d:\frozen
frozenTimePeriodInSecs = 10
rotatePeriodInSecs = 10
maxHotIdleSecs = 180
Tags (1)
0 Karma

yannK
Splunk Employee
Splunk Employee

the time retention is controled by frozenTimePeriodInSecs, so for 90 days = 60*60*24*90= 7 776 000 seconds

frozenTimePeriodInSecs=7776000

Please read this http://wiki.splunk.com/Deploy:BucketRotationAndRetention before tuning rotatePeriodInSecs and maxHotIdleSecs they control the hot to warm rotation.
Eventually you could use maxHotIdleSecs=86400 to reduce the span of your hot buckets to 1 day per bucket.

yannK
Splunk Employee
Splunk Employee

oh, sorry, I meant maxHotSpanSecs not maxHotIdleSecs

I usually use maxHotIdleSecs as my time span for my hot buckets, to force them to roll more often.

0 Karma

jichen
Explorer

Per this document http://wiki.splunk.com/Deploy:BucketRotationAndRetention, I add a line "maxHotSpanSecs = 200",the data should be retired after 200+10 Secs, but actually it archived the data at below time point: 11:06,12:03,13:03,13:12,14:04,15:03,15:07. So I don't know when it will do the archive job and how to control the retention of the data.

0 Karma

jichen
Explorer

Hi, as I mentioned, when I test with below setting
frozenTimePeriodInSecs = 10
rotatePeriodInSecs = 10
maxHotIdleSecs = 200
, it maybe not delete and archive the data at all.

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...