Monitoring Splunk

Multiple indexes + disk space usage

skippylou
Communicator

So if I follow the data space and retirement process correctly, it works in a circular manner with old data being deleted (as its frozen) as needed to make room for new data. With the default setting of 500GB max index size.

My question would be say you had two indexes, index1 with a max size setting of 400GB and index2 with a max size setting of 200GB, and you have 400GB of usable space. Let's say at some point you have 200GB used in index1 and you hit 200GB on index2. So index2 should be working in a circular fashion at this point rotating old data out. But what happens with index1? It is at only half its max, but the disk is full. Does it do circular based on this or does no old data get rotated out until an age-based policy applies? Should the total max index sizes never equal more than your total disk space available?

What is the best way to manage max index sizes on multiple indexes and also to ensure that any new data coming in regardless of the index makes it into its index?

Thanks,

Scott

Tags (2)
1 Solution

Genti
Splunk Employee
Splunk Employee

Scott,

Yes, total max index sizes should never equal more then your total disk space available.
In your described scenario, when diskspace is 400GB and Index1 has 200GB and index2 has also 200GB indexing will be stopped until you release at least 2GB(definitely more then 2 though) space.

Best way would be: Index A + B + C + D +... < TOTAL HD Space

View solution in original post

Genti
Splunk Employee
Splunk Employee

Scott,

Yes, total max index sizes should never equal more then your total disk space available.
In your described scenario, when diskspace is 400GB and Index1 has 200GB and index2 has also 200GB indexing will be stopped until you release at least 2GB(definitely more then 2 though) space.

Best way would be: Index A + B + C + D +... < TOTAL HD Space

skippylou
Communicator

Good to know, thanks Genti!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...