Deployment Architecture

If maxDataSize in indexes.conf is for hot buckets, then what parameter needs to be defined to set the size for warm and cold buckets?

ankithreddy777
Contributor

In indexes.conf, it is given that "maxDataSize: The maximum size in MB for a hot DB to reach before a roll to warm is triggered". What parameter is defined to set size for warm buckets and the cold buckets?

Labels (1)
0 Karma
1 Solution

TStrauch
Communicator

Hi,

you have multiple configuration parameter for roling buckets.

Hot to Warm:

maxHotBuckets=<integer>
maxHotSpanSecs=<integer>

Hot/Warm to Cold:

homepath.maxDataSizeMB=<integer> |this is the max Data size for hot and warm db

Warm to Cold:

maxwarmDBCount=<integer>

Cold to Frozen:

coldPath.maxDataSize=<integer>

And the last is for the retention on the index

maxTotalDataSizeMB=<integer>
frozenTimePeriodInSecs=<integer>

Check this paramenter on the indexes.conf Documentation.

Hope this helps

kind regards

View solution in original post

TStrauch
Communicator

Hi,

you have multiple configuration parameter for roling buckets.

Hot to Warm:

maxHotBuckets=<integer>
maxHotSpanSecs=<integer>

Hot/Warm to Cold:

homepath.maxDataSizeMB=<integer> |this is the max Data size for hot and warm db

Warm to Cold:

maxwarmDBCount=<integer>

Cold to Frozen:

coldPath.maxDataSize=<integer>

And the last is for the retention on the index

maxTotalDataSizeMB=<integer>
frozenTimePeriodInSecs=<integer>

Check this paramenter on the indexes.conf Documentation.

Hope this helps

kind regards

View solution in original post

ankithreddy777
Contributor

Thank you for your reply. Is there any way to set maximum size of warm bucket similar to hot bucket. Because homepath.maxDataSizeMB=0 by default it suggest there are no constraints on homepath size. what max size each of 300(by default) warm buckets take .

0 Karma

skramp
Engager

Hello, I think coldPath.maxDataSize=INTEGER might be coldPath.maxDataSizeMB=INTEGER. There is a maxDataSize without MB but this is for hot to warm.

0 Karma

TStrauch
Communicator

The maximum size of a warm bucket must be the maximum size of a hot bucket. The difference between hot and warm buckets is, that a hot bucket is Open for write Operations. After the hot bucket is full It is rotated to a non writable only readable warm bucket. As you see you define the warm buckets through your hot buckets.

Is this what you wanted to know?

0 Karma

ankithreddy777
Contributor

Yup, I got the complete picture. Thank you so much. You are the best. May I no for what configuration changes the indexer restarts ,. I am confused because for congurations changes like creating indexes using indexes.conf and pushing those changes to indexes using CM do not result in indexers restart.

0 Karma

TStrauch
Communicator

Just look at the following link. There is explained which Changes generate a restart or a reload of the indexer Cluster Peers.

http://docs.splunk.com/Documentation/Splunk/6.5.0/Indexer/Updatepeerconfigurations

Kind regards

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!