Getting Data In

Need to set an upper size limit across multiple volumes

jducote
New Member

I need to set an upper size limit across multiple volumes, for indexing, regardless of hot/warm/cold status. It is my understanding that I may do this by setting the homePath.maxDataSizeMB switch in indexes.conf. Do I need to establish the three volumes individually, or will setting (in example) homePath.maxDataSizeMB=850000[MB] limit each registered volume as having an upper limit of 850GB? Or do I need to specify the limit for each volume, in the same file?

Some background:

3 Volumes of 1TB (986GB usable):
/dev/md127 985G 897G 38G 96% /opt/splunk
/dev/md125 985G 394G 541G 43% /opt/splunk-index
/dev/md126 985G 240G 695G 26% /opt/splunk-index2

As you can see, we are nearly full on the first volume. Pruning index size was successful, but mostly affected /opt/splunk-index. My desired state would be to have data more evenly distributed amongst the three volumes.

0 Karma

yannK
Splunk Employee
Splunk Employee

If you want to control the size per volume instead of per index, check this guide
http://docs.splunk.com/Documentation/Splunk/latest/Indexer/Configureindexstoragesize

The other way is more delicate, you can be very specific for each index using the number of hot and warm buckets to spread the buckets between homePath and coldPath.
http://wiki.splunk.com/Deploy:BucketRotationAndRetention

0 Karma

jducote
New Member

Thanks, it looks like using the homePath.maxDataSizeMB=X will be the most elegant option, per:
"Space taken by the whole index
The maximum size of all buckets for an index is maxTotalDataSizeMB. This is the size of hot+warm+cold. If you have hot+warm and cold on different volumes, try to maximize the use of the space." (http://wiki.splunk.com/Deploy:BucketRotationAndRetention)

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...