Monitoring Splunk

Multiple indexes + disk space usage

skippylou
Communicator

So if I follow the data space and retirement process correctly, it works in a circular manner with old data being deleted (as its frozen) as needed to make room for new data. With the default setting of 500GB max index size.

My question would be say you had two indexes, index1 with a max size setting of 400GB and index2 with a max size setting of 200GB, and you have 400GB of usable space. Let's say at some point you have 200GB used in index1 and you hit 200GB on index2. So index2 should be working in a circular fashion at this point rotating old data out. But what happens with index1? It is at only half its max, but the disk is full. Does it do circular based on this or does no old data get rotated out until an age-based policy applies? Should the total max index sizes never equal more than your total disk space available?

What is the best way to manage max index sizes on multiple indexes and also to ensure that any new data coming in regardless of the index makes it into its index?

Thanks,

Scott

Tags (2)
1 Solution

Genti
Splunk Employee
Splunk Employee

Scott,

Yes, total max index sizes should never equal more then your total disk space available.
In your described scenario, when diskspace is 400GB and Index1 has 200GB and index2 has also 200GB indexing will be stopped until you release at least 2GB(definitely more then 2 though) space.

Best way would be: Index A + B + C + D +... < TOTAL HD Space

View solution in original post

Genti
Splunk Employee
Splunk Employee

Scott,

Yes, total max index sizes should never equal more then your total disk space available.
In your described scenario, when diskspace is 400GB and Index1 has 200GB and index2 has also 200GB indexing will be stopped until you release at least 2GB(definitely more then 2 though) space.

Best way would be: Index A + B + C + D +... < TOTAL HD Space

skippylou
Communicator

Good to know, thanks Genti!

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...