topic Re: Time based retention time, without a thawed path in Getting Data In

Time based retention time, without a thawed path

avoelk — Mon, 07 Oct 2024 13:57:08 GMT

I'm trying to configure the indexes.conf in such a way that its data retention is exactly 180 days and then does NOT get frozen, but gets deleted.

I've tried to set it with frozenTimePeriodInSecs = 15552000 but now I get the following error:

Validation errors are present in the bundle. Errors=peer=XXX, stanza=someidx Required parameter=thawedPath not configured;

so I HAVE TO put a thawed path in it even tho I don't want to freeze anything? how does that make sense?

Kind regards for a clarification!

Re: Time based retention time, without a thawed path

gcusello — Mon, 07 Oct 2024 14:10:44 GMT

Hi @avoelk,

yes, it's a required parameter even if you don't want to restore thawed buckets.

Remember in Splunk the retention period is managed at bucket level, in other words, a bucket is deleted (or frozen) only when the latest event is older than the retention period, this means that you'll surely have in your buckets events older than the retention period, because they are in a bucket with younger events.

Ciao.

Giuseppe

Re: Time based retention time, without a thawed path

avoelk — Mon, 07 Oct 2024 14:42:18 GMT

Hi Giuseppe and thanks for the swift answer!

But how does it behave if I don't want to allocate a specific diskspace for thawed files/frozen files?

So there is no way to just have a retention of 180 days and afterwards it will be deleted or did I get something of your answer wrong?

Kind regards

Re: Time based retention time, without a thawed path

gcusello — Mon, 07 Oct 2024 14:59:22 GMT

Hi @avoelk ,

you don't need to allocate any disk space: the thawed path is only a mount point that you can use to recover frozen buckets, if you don't need it, you must only define the mount point (the thawed_path) in indexes.conf and then you don't need to allocate any disk space.

Ciao.

Giuseppe

Re: Time based retention time, without a thawed path

PickleRick — Mon, 07 Oct 2024 15:05:29 GMT

Thawed path is the directory in which you'd have to manually put the data to be thawed (or where Splunk puts it after thawing; I don't remember I don't generally thaw buckets). It doesn't have anything to do with the freezing process. If you don't define frozen path (and freeze script) the data will get deleted when rolled to frozen.

And be aware of what @gcusello said - data is rolled on a per bucket basis which means that "resolution" of the bucket rolling process depends on the contents of the buckets - data is being rolled to frozen when _newest_ event in a bucket is older than the retention period. That can be important especially in case of quarantine buckets.

Re: Time based retention time, without a thawed path

avoelk — Mon, 07 Oct 2024 15:06:21 GMT

Thanks a lot !