Monitoring Splunk

Multiple indexes + disk space usage

skippylou
Communicator

So if I follow the data space and retirement process correctly, it works in a circular manner with old data being deleted (as its frozen) as needed to make room for new data. With the default setting of 500GB max index size.

My question would be say you had two indexes, index1 with a max size setting of 400GB and index2 with a max size setting of 200GB, and you have 400GB of usable space. Let's say at some point you have 200GB used in index1 and you hit 200GB on index2. So index2 should be working in a circular fashion at this point rotating old data out. But what happens with index1? It is at only half its max, but the disk is full. Does it do circular based on this or does no old data get rotated out until an age-based policy applies? Should the total max index sizes never equal more than your total disk space available?

What is the best way to manage max index sizes on multiple indexes and also to ensure that any new data coming in regardless of the index makes it into its index?

Thanks,

Scott

Tags (2)
1 Solution

Genti
Splunk Employee
Splunk Employee

Scott,

Yes, total max index sizes should never equal more then your total disk space available.
In your described scenario, when diskspace is 400GB and Index1 has 200GB and index2 has also 200GB indexing will be stopped until you release at least 2GB(definitely more then 2 though) space.

Best way would be: Index A + B + C + D +... < TOTAL HD Space

View solution in original post

Genti
Splunk Employee
Splunk Employee

Scott,

Yes, total max index sizes should never equal more then your total disk space available.
In your described scenario, when diskspace is 400GB and Index1 has 200GB and index2 has also 200GB indexing will be stopped until you release at least 2GB(definitely more then 2 though) space.

Best way would be: Index A + B + C + D +... < TOTAL HD Space

skippylou
Communicator

Good to know, thanks Genti!

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...