How to fix Splunk indexers max volume size exceeded?

[avivfri]

Hello

I have 5 indexers managed by Cluster Master.

On the indexes.conf (located as master-app) I have the following configuration:

[default]
maxTotalDataSizeMB = 1000000
frozenTimePeriodInSecs = 13824000

[volume:hot]
path = /hot/splunk_db/
maxVolumeDataSizeMB = 2800000

from my understating, the Hot volume on each indexer should not be more than 2.8TB. but, actually the volume exceeded this limit and reached 2.9TB. 

can someone please assist?

Thank you

[gcusello · Accepted solution]

Hi @avivfri,

remember that the passages from Hot to Warm, from Warm to Cold and from Cold to Frozen is always done at bucket level, this means that a full bucket passes from a state to another, not the single event data.

For the same reason it's normal to have data that exceeds the retention period, because they are in a buckets containing events in the retention period.

In other words, a bucket passes from Hot to Warm when all the events in the bucket exceeds the defined retention period or when the bucket is closed so it can change state.

In your case, probably you have buckets of 9.x GB, so they don't change status.

You could change the dimension of the buckets but I usually prefer to leave the default values.

Ciao.

Giuseppe

[gcusello]

Hi @avivfri,

good for you, see next time!
Ciao and happy splunking
Giuseppe
P.S.: Karma Points are appreciated 😉

[avivfri]

Hi Giuseppe

Thank you for the detailed answer!

so you are saying that "frozenTimePeriodInSecs" takes priority over "maxVolumeDataSizeMB"?

Thank you

[gcusello]

Hi @avivfri,

no there isn't any priority, I'm only saying that a full bucket rolls to another status, this means that the dimensions could exceed the configured limit.

Ciao.

Giuseppe