Deployment Architecture

how to use maxDataSize to control the size of each bucket

jek01
New Member

I use maxDataSize to control the size of each bucket.
The purpose is to limit each bucket up to 6MB.
Join "maxDataSize = 6 maxHotBuckets = 1" in test of local/indexes.conf.
(index=test)
But the result is a failure.
Execution results:
1. size of file=5MB ,Execution results:2 hot db,no any warm db. BUT Expected: only one bucket.
2. size of file=10MB ,Execution results:1 hot db,no any warm db. BUT Expected: only two buckets.

excuse me, can you tell me the reason? AND how can I achieve the goals?
TKS.

0 Karma
1 Solution

jkat54
SplunkTrust
SplunkTrust

I believe your answer lies in this excerpt from indexes.conf:

rotatePeriodInSecs = <positive integer>
* Controls the service period (in seconds): how often splunkd performs
  certain housekeeping tasks.  Among these tasks are:
  * Check if a new hotdb needs to be created.
  * Check if there are any cold DBs that should be frozen.
  * Check whether buckets need to be moved out of hot and cold DBs, due to
    respective size constraints (i.e., homePath.maxDataSizeMB and
    coldPath.maxDataSizeMB)
* This value becomes the default value of the rotatePeriodInSecs attribute
  for all volumes (see rotatePeriodInSecs in the Volumes section)
* Highest legal value is 4294967295
* Defaults to 60.

Splunk is making its best effort to give you 6mb per bucket, but it's only checking to see if it needs to rotate buckets every minute. You could reduce this time period but i recommend a maxDataSize of at least 100. Why are you trying to make the buckets so small?

View solution in original post

jkat54
SplunkTrust
SplunkTrust

I believe your answer lies in this excerpt from indexes.conf:

rotatePeriodInSecs = <positive integer>
* Controls the service period (in seconds): how often splunkd performs
  certain housekeeping tasks.  Among these tasks are:
  * Check if a new hotdb needs to be created.
  * Check if there are any cold DBs that should be frozen.
  * Check whether buckets need to be moved out of hot and cold DBs, due to
    respective size constraints (i.e., homePath.maxDataSizeMB and
    coldPath.maxDataSizeMB)
* This value becomes the default value of the rotatePeriodInSecs attribute
  for all volumes (see rotatePeriodInSecs in the Volumes section)
* Highest legal value is 4294967295
* Defaults to 60.

Splunk is making its best effort to give you 6mb per bucket, but it's only checking to see if it needs to rotate buckets every minute. You could reduce this time period but i recommend a maxDataSize of at least 100. Why are you trying to make the buckets so small?

adonio
Ultra Champion

i think @jkat54 meant indexes.conf and not props.conf

jkat54
SplunkTrust
SplunkTrust

Thanks that's what I meant but I couldn't edit until just now because my account was locked.

0 Karma

jek01
New Member

Hi ... Thanks
Because I take a small data test, so every bucket is so small. I'll try again.

0 Karma
Get Updates on the Splunk Community!

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...

Security Highlights | January 2023 Newsletter

January 2023 Splunk Security Essentials (SSE) 3.7.0 ReleaseThe free Splunk Security Essentials (SSE) 3.7.0 app ...

Platform Highlights | January 2023 Newsletter

 January 2023Peace on Earth and Peace of Mind With Business ResilienceAll organizations can start the new year ...