Deployment Architecture

If maxDataSize in indexes.conf is for hot buckets, then what parameter needs to be defined to set the size for warm and cold buckets?

ankithreddy777
Contributor

In indexes.conf, it is given that "maxDataSize: The maximum size in MB for a hot DB to reach before a roll to warm is triggered". What parameter is defined to set size for warm buckets and the cold buckets?

Labels (1)
0 Karma
1 Solution

TStrauch
Communicator

Hi,

you have multiple configuration parameter for roling buckets.

Hot to Warm:

maxHotBuckets=<integer>
maxHotSpanSecs=<integer>

Hot/Warm to Cold:

homepath.maxDataSizeMB=<integer> |this is the max Data size for hot and warm db

Warm to Cold:

maxwarmDBCount=<integer>

Cold to Frozen:

coldPath.maxDataSize=<integer>

And the last is for the retention on the index

maxTotalDataSizeMB=<integer>
frozenTimePeriodInSecs=<integer>

Check this paramenter on the indexes.conf Documentation.

Hope this helps

kind regards

View solution in original post

TStrauch
Communicator

Hi,

you have multiple configuration parameter for roling buckets.

Hot to Warm:

maxHotBuckets=<integer>
maxHotSpanSecs=<integer>

Hot/Warm to Cold:

homepath.maxDataSizeMB=<integer> |this is the max Data size for hot and warm db

Warm to Cold:

maxwarmDBCount=<integer>

Cold to Frozen:

coldPath.maxDataSize=<integer>

And the last is for the retention on the index

maxTotalDataSizeMB=<integer>
frozenTimePeriodInSecs=<integer>

Check this paramenter on the indexes.conf Documentation.

Hope this helps

kind regards

ankithreddy777
Contributor

Thank you for your reply. Is there any way to set maximum size of warm bucket similar to hot bucket. Because homepath.maxDataSizeMB=0 by default it suggest there are no constraints on homepath size. what max size each of 300(by default) warm buckets take .

0 Karma

skramp
Path Finder

Hello, I think coldPath.maxDataSize=INTEGER might be coldPath.maxDataSizeMB=INTEGER. There is a maxDataSize without MB but this is for hot to warm.

0 Karma

TStrauch
Communicator

The maximum size of a warm bucket must be the maximum size of a hot bucket. The difference between hot and warm buckets is, that a hot bucket is Open for write Operations. After the hot bucket is full It is rotated to a non writable only readable warm bucket. As you see you define the warm buckets through your hot buckets.

Is this what you wanted to know?

0 Karma

ankithreddy777
Contributor

Yup, I got the complete picture. Thank you so much. You are the best. May I no for what configuration changes the indexer restarts ,. I am confused because for congurations changes like creating indexes using indexes.conf and pushing those changes to indexes using CM do not result in indexers restart.

0 Karma

TStrauch
Communicator

Just look at the following link. There is explained which Changes generate a restart or a reload of the indexer Cluster Peers.

http://docs.splunk.com/Documentation/Splunk/6.5.0/Indexer/Updatepeerconfigurations

Kind regards

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...