Getting Data In

Why is my Index Current Size greater than my Max Size (Cluster)

richardgosnay
Explorer

Hi Splunk Folk,

I've spent most of the morning trying to find this with no luck, I've seen some similar posts but none of the solutions work for me.  Why is the "Current Size" greater than the "Max Size" for several indexes which reside on a cluster?

richardgosnay_0-1620649278832.png

Here is an example of the indexes.conf file that my master is pushing out for a 100Gb Max Size index.

[index_name]
homePath = $SPLUNK_DB\index_name\db
coldPath = $SPLUNK_DB\index_name\colddb
thawedPath = $SPLUNK_DB\index_name\thaweddb
repFactor = auto
enableDataIntegrityControl = 0
enableTsidxReduction = 0
maxTotalDataSizeMB = 102400
bucketRebuildMemoryHint = 0
compressRawdata = 1
enableOnlineBucketRepair = 1
minHotIdleSecsBeforeForceRoll = 0
suspendHotRollByDeleteQuery = 0
syncMeta = 1
disabled = 0

 

I haven't tried changing the maxTotalDataSizeMB value and pushing out new configs yet because I wanted to understand why it's doing this in the first place?

 

Any ideas?

Labels (2)
Tags (1)
0 Karma

richardgosnay
Explorer

I have now pushed new configs with updated maxTotalDataSizeMB and maxDataSizeMB for both home and cold path, still the Current Size is greater and shows no signs of reducing.

I have two environments, one in Belgium and one in Australia, each are installed almost identically.

The Australian one works perfectly, all indexes are kept below their max size settings, however the Belgium one refuses to budge.  Am I missing something? Is there a setting I don't know about?

Any suggest would be greatly appreciated.

Thank you

 

Here is an example of an index I've capped at 30Gb (10Gb for hot/warm and 20Gb for cold)

[index-1]
homePath = $SPLUNK_DB\index-1\db
coldPath = $SPLUNK_DB\index-1\colddb
thawedPath = $SPLUNK_DB\index-1\thaweddb
repFactor = auto
enableDataIntegrityControl = 0
enableTsidxReduction = 0
maxTotalDataSizeMB = 30720
homePath.maxDataSizeMB = 10240
coldPath.maxDataSizeMB = 20480
bucketRebuildMemoryHint = 0
compressRawdata = 1
enableOnlineBucketRepair = 1
minHotIdleSecsBeforeForceRoll = 0
rtRouterQueueSize =
rtRouterThreads =
suspendHotRollByDeleteQuery = 0
syncMeta = 1
disabled = 0

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...