Getting Data In

Why is my Index Current Size greater than my Max Size (Cluster)

richardgosnay
Explorer

Hi Splunk Folk,

I've spent most of the morning trying to find this with no luck, I've seen some similar posts but none of the solutions work for me.  Why is the "Current Size" greater than the "Max Size" for several indexes which reside on a cluster?

richardgosnay_0-1620649278832.png

Here is an example of the indexes.conf file that my master is pushing out for a 100Gb Max Size index.

[index_name]
homePath = $SPLUNK_DB\index_name\db
coldPath = $SPLUNK_DB\index_name\colddb
thawedPath = $SPLUNK_DB\index_name\thaweddb
repFactor = auto
enableDataIntegrityControl = 0
enableTsidxReduction = 0
maxTotalDataSizeMB = 102400
bucketRebuildMemoryHint = 0
compressRawdata = 1
enableOnlineBucketRepair = 1
minHotIdleSecsBeforeForceRoll = 0
suspendHotRollByDeleteQuery = 0
syncMeta = 1
disabled = 0

 

I haven't tried changing the maxTotalDataSizeMB value and pushing out new configs yet because I wanted to understand why it's doing this in the first place?

 

Any ideas?

Labels (2)
Tags (1)
0 Karma

richardgosnay
Explorer

I have now pushed new configs with updated maxTotalDataSizeMB and maxDataSizeMB for both home and cold path, still the Current Size is greater and shows no signs of reducing.

I have two environments, one in Belgium and one in Australia, each are installed almost identically.

The Australian one works perfectly, all indexes are kept below their max size settings, however the Belgium one refuses to budge.  Am I missing something? Is there a setting I don't know about?

Any suggest would be greatly appreciated.

Thank you

 

Here is an example of an index I've capped at 30Gb (10Gb for hot/warm and 20Gb for cold)

[index-1]
homePath = $SPLUNK_DB\index-1\db
coldPath = $SPLUNK_DB\index-1\colddb
thawedPath = $SPLUNK_DB\index-1\thaweddb
repFactor = auto
enableDataIntegrityControl = 0
enableTsidxReduction = 0
maxTotalDataSizeMB = 30720
homePath.maxDataSizeMB = 10240
coldPath.maxDataSizeMB = 20480
bucketRebuildMemoryHint = 0
compressRawdata = 1
enableOnlineBucketRepair = 1
minHotIdleSecsBeforeForceRoll = 0
rtRouterQueueSize =
rtRouterThreads =
suspendHotRollByDeleteQuery = 0
syncMeta = 1
disabled = 0

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...