topic Ideal indexes.conf in Getting Data In

Ideal indexes.conf

nawazns5038 — Fri, 09 Mar 2018 01:47:47 GMT

Hi,

Can I please know the ideal configurations for indexes.conf ?

Should we include parameters like homePath.maxDataSizeMB , coldPath.maxDataSizeMB etc. ?

Or is it enough to specify only frozenTimePeriodInSecs

Re: Ideal indexes.conf

damiensurat — Fri, 09 Mar 2018 03:16:37 GMT

This is a hard question ion to answer as the parameters you mention should be set to accommodate data size and retention time of your data. Being that data in Splunk index buckets will start to roll from hot to warm to cold and then frozen based on both the size and /or retention time I would suggest that you understand how much data you need to capture and retain as well as how long the data should remain available in the index. You can always increase / decrease the size and retention time as needed.

Re: Ideal indexes.conf

deepashri_123 — Fri, 09 Mar 2018 04:05:28 GMT

Hey nawazns5038,

Your total retention period includes hot warm and cold buckets. There are default parameters in splunk from rolling hot to warm buckets and warm to cold, although can be changed. However frozenTimePeriodInSecs parameter decides when the bucket has to be deleted from Splunk, so adding this parameter should do the trick.

You can refer the doc below:
http://wiki.splunk.com/Deploy:BucketRotationAndRetention

Let me know if this helps!!

Re: Ideal indexes.conf

nawazns5038 — Tue, 29 Sep 2020 18:26:09 GMT

Hi Deepa

homePath.maxDataSizeMB = 2000000
coldPath.maxDataSizeMB = 15664000
frozenTimePeriodInSecs = 2505600
maxTotalDataSizeMB = 34664000
maxDataSize = auto_high_volume
repFactor = auto

How does Splunk roll the buckets from hot to warm warm to cold based on the settings I mentioned above.

Re: Ideal indexes.conf

nawazns5038 — Tue, 13 Mar 2018 21:16:51 GMT

If I just mention frozenTimePeriodInSecs as 30 days , how does Splunk roll the buckets exactly so that the data gets deleted in 30 days ?

Will adding the parameters like homePath.maxDataSizeMB , coldPath.maxDataSizeMB effect the rolling of buckets ?

Re: Ideal indexes.conf

deepashri_123 — Wed, 14 Mar 2018 06:19:23 GMT

You can refer this link to understand how bucket ages:
http://docs.splunk.com/Documentation/Splunk/7.0.2/Indexer/HowSplunkstoresindexes

When you mention frozenTimePeriodInSecs, this parameter decides the retention of bucket from coldpath and deletes the file from splunk

Re: Ideal indexes.conf

nawazns5038 — Fri, 16 Mar 2018 23:57:22 GMT

It does not mention the way how Splunk rolls the buckets if we just mention the frozenTimePeriodInSecs .

The question is pretty straight forward, we set frozenTimePeriodInSecs = 30 days .... how does splunk roll the buckets so that data gets deleted by 30 days from the day it comes in .

you have mentioned that the "When you mention frozenTimePeriodInSecs, this parameter decides the retention of bucket from coldpath and deletes the file from splunk "
but there are stages in between which the data goes through to reach cold state.

Re: Ideal indexes.conf

nawazns5038 — Sat, 17 Mar 2018 00:03:58 GMT

What if the data volume is low and is present only in the hot and warm buckets itself and didn't come into cold still and the period has exceeded 30 days ?
So the retention policy will not apply in this case ??

Re: Ideal indexes.conf

ddrillic — Sat, 17 Mar 2018 01:38:51 GMT

The two important ones are frozenTimePeriodInSecs and maxTotalDataSizeMB with the default of 1/2 TB.