How to set the bucket retention policy for 2 months before automatically deleting the data?

[sbenamro]

Hi,

I'd like to set the Bucket time to keep data for 2 months and then to automatically remove it.

how do I do it ?

[NOUMSSI]

Hi,

Here is an example to show you how to set it:

[default]
maxWarmDBCount = 200
frozenTimePeriodInSecs = 432000
rotatePeriodInSecs = 30
coldToFrozenScript = "$SPLUNK_HOME/bin/python"
"$SPLUNK_HOME/bin/myColdToFrozenScript.py"

I don't advice you to set this attribut: maxHotSpanSecs
By defaults it's set to 7776000 seconds (90 days).
NOTE: If you set this too small, you can get an explosion of hot/warm

[martin_mueller · Accepted solution]

The key settings for short retention times are these two indexes.conf:

frozenTimePeriodInSecs = <nonnegative integer>
maxHotSpanSecs = <positive integer>

The first setting defaults to six years, you'll want to set that to two months so 5270400 (61 days) or however long your months are in seconds. The second setting defaults to 90 days, which is great for six years of retention but not great for two months. A bucket is removed when its youngest event crosses the frozenTimePeriodInSecs, so you would effectively wait five months. Set this to a week or so to actually remove your data close to those two months.

See http://docs.splunk.com/Documentation/Splunk/6.2.2/admin/indexesconf for reference.

[martin_mueller]

That's an internal space Splunk uses to store block signatures. Normally you do not change the configuration for it.

[martin_mueller]

Every settings exists once for every index, the one under [_internal] applies to the _internal index while the one under [default] applies to all indexes that don't set their own value. You should modify the settings for the indexes you want to modify.

[sbenamro]

thanks for your help ! much appreciated.
one last question - what is the _blocksignature ?

[sbenamro]

Thanks for your help.
I'm using only the file over the local folder.
yet - I've noticed that I have "maxHotSpanSecs " in several places in that file, which are [default] and [_internal]

and I've noticed that "frozenTimePeriodInSecs " exist in several places as well which are - [default] and [_blocksignature] and [_internal] and [_introspection] and [_thefishbucket] and [history]

so where should I change it ?

[martin_mueller]

You should set this for the index you want to modify in the local directory of that index' app, or in system/local if the index is defined in system. Do not modify .conf files in any default directory.
Which index are you trying to modify?

[sbenamro]

thanks for the answer but under which section should I change it ?
I have for example - [_internal] or [default]