topic How to set the bucket retention policy for 2 months before automatically deleting the data? in Deployment Architecture

How to set the bucket retention policy for 2 months before automatically deleting the data?

sbenamro — Sun, 12 Apr 2015 14:39:34 GMT

Hi,

I'd like to set the Bucket time to keep data for 2 months and then to automatically remove it.

how do I do it ?

Re: How to set the bucket retention policy for 2 months before automatically deleting the data?

martin_mueller — Sun, 12 Apr 2015 16:52:19 GMT

The key settings for short retention times are these two indexes.conf:

frozenTimePeriodInSecs = <nonnegative integer>
maxHotSpanSecs = <positive integer>

The first setting defaults to six years, you'll want to set that to two months so 5270400 (61 days) or however long your months are in seconds. The second setting defaults to 90 days, which is great for six years of retention but not great for two months. A bucket is removed when its youngest event crosses the frozenTimePeriodInSecs, so you would effectively wait five months. Set this to a week or so to actually remove your data close to those two months.

See http://docs.splunk.com/Documentation/Splunk/6.2.2/admin/indexesconf for reference.

Re: How to set the bucket retention policy for 2 months before automatically deleting the data?

sbenamro — Mon, 13 Apr 2015 07:57:18 GMT

thanks for the answer but under which section should I change it ?
I have for example - [_internal] or [default]

Re: How to set the bucket retention policy for 2 months before automatically deleting the data?

NOUMSSI — Mon, 13 Apr 2015 08:32:28 GMT

Hi,

Here is an example to show you how to set it:

[default]
maxWarmDBCount = 200
frozenTimePeriodInSecs = 432000
rotatePeriodInSecs = 30
coldToFrozenScript = "$SPLUNK_HOME/bin/python"
"$SPLUNK_HOME/bin/myColdToFrozenScript.py"

I don't advice you to set this attribut: maxHotSpanSecs
By defaults it's set to 7776000 seconds (90 days).
NOTE: If you set this too small, you can get an explosion of hot/warm

Re: How to set the bucket retention policy for 2 months before automatically deleting the data?

martin_mueller — Mon, 13 Apr 2015 11:31:55 GMT

You should set this for the index you want to modify in the local directory of that index' app, or in system/local if the index is defined in system. Do not modify .conf files in any default directory.
Which index are you trying to modify?

Re: How to set the bucket retention policy for 2 months before automatically deleting the data?

sbenamro — Mon, 28 Sep 2020 19:31:44 GMT

Thanks for your help.
I'm using only the file over the local folder.
yet - I've noticed that I have "maxHotSpanSecs " in several places in that file, which are [default] and [_internal]

and I've noticed that "frozenTimePeriodInSecs " exist in several places as well which are - [default] and [_blocksignature] and [_internal] and [_introspection] and [_thefishbucket] and [history]

so where should I change it ?

Re: How to set the bucket retention policy for 2 months before automatically deleting the data?

martin_mueller — Tue, 14 Apr 2015 10:06:02 GMT

Every settings exists once for every index, the one under [_internal] applies to the _internal index while the one under [default] applies to all indexes that don't set their own value. You should modify the settings for the indexes you want to modify.

Re: How to set the bucket retention policy for 2 months before automatically deleting the data?

sbenamro — Tue, 14 Apr 2015 11:08:56 GMT

thanks for your help ! much appreciated.
one last question - what is the _blocksignature ?

Re: How to set the bucket retention policy for 2 months before automatically deleting the data?

martin_mueller — Tue, 14 Apr 2015 11:15:19 GMT

That's an internal space Splunk uses to store block signatures. Normally you do not change the configuration for it.