topic Re: Ask for Indexes.conf Configuration - Bucket Rotation- in Deployment Architecture

Ask for Indexes.conf Configuration - Bucket Rotation-

AliMaher — Wed, 13 Nov 2024 22:44:49 GMT

Hello Splunker,

I have two volumes with the following specs:

Hot/Warm Volume: 5.25 TB Cold Volume: 4.75 TB ================================ [volume:hot] path = /opt/splunk-hwdata maxVolumeDataSizeMB = 7602176 [volume:cold] path = /opt/splunk-Colddata maxVolumeDataSizeMB = 4980736 ================================== [Win] repFactor = auto homePath = volume:hot/$_index_name/db coldPath = volume:cold/$_index_name/colddb thawedPath = /opt/splunk-Colddata/$_index_name/thaweddb homePath.maxDataSizeMB = 7602176 coldPath.maxDataSizeMB = 4980736 maxWarmDBCount = 720 frozenTimePeriodInSecs = 5184000 maxDataSize = auto_high_volume [FW] repFactor = auto homePath = volume:hot/$_index_name/db coldPath = volume:cold/$_index_name/colddb thawedPath = /opt/splunk-Colddata/$_index_name/thaweddb homePath.maxDataSizeMB = 7602176 coldPath.maxDataSizeMB = 4980736 maxWarmDBCount = 720 frozenTimePeriodInSecs = 5184000 maxDataSize = auto_high_volume

====================================

Notice we have re-configured the below:

[diskUsage] minFreeSpace = 20000

Finally, we have reached the bottom of the question 😀.

I am doubt if this configuration can maintain the below requirements:

The data retention period for the online data is 2 months.

- Hot/Warm – 1 month - Cold – 1 month

Re: Ask for Indexes.conf Configuration - Bucket Rotation-

AliMaher — Wed, 13 Nov 2024 22:19:23 GMT

Reply:

Can we enforce the data to be rolled from the Hot/warm to Cold after one month then from Cold to frozen after one month.

Re: Ask for Indexes.conf Configuration - Bucket Rotation-

isoutamo — Wed, 13 Nov 2024 23:51:24 GMT

shortly Nope.

There are quite many answers which this has already discussed earlier.

Main point here is that bucket is manage by youngest event inside it. As there are several bucket which _time can differ heavily to each other _time:s in that bucket, you cannot get exactly 1 month time period in hot+warm+cold. It's always defined by combination of several parameters. You can found those from older answers or docs.

r. Ismo