Getting Data In

Need to set an upper size limit across multiple volumes

jducote
New Member

I need to set an upper size limit across multiple volumes, for indexing, regardless of hot/warm/cold status. It is my understanding that I may do this by setting the homePath.maxDataSizeMB switch in indexes.conf. Do I need to establish the three volumes individually, or will setting (in example) homePath.maxDataSizeMB=850000[MB] limit each registered volume as having an upper limit of 850GB? Or do I need to specify the limit for each volume, in the same file?

Some background:

3 Volumes of 1TB (986GB usable):
/dev/md127 985G 897G 38G 96% /opt/splunk
/dev/md125 985G 394G 541G 43% /opt/splunk-index
/dev/md126 985G 240G 695G 26% /opt/splunk-index2

As you can see, we are nearly full on the first volume. Pruning index size was successful, but mostly affected /opt/splunk-index. My desired state would be to have data more evenly distributed amongst the three volumes.

0 Karma

yannK
Splunk Employee
Splunk Employee

If you want to control the size per volume instead of per index, check this guide
http://docs.splunk.com/Documentation/Splunk/latest/Indexer/Configureindexstoragesize

The other way is more delicate, you can be very specific for each index using the number of hot and warm buckets to spread the buckets between homePath and coldPath.
http://wiki.splunk.com/Deploy:BucketRotationAndRetention

0 Karma

jducote
New Member

Thanks, it looks like using the homePath.maxDataSizeMB=X will be the most elegant option, per:
"Space taken by the whole index
The maximum size of all buckets for an index is maxTotalDataSizeMB. This is the size of hot+warm+cold. If you have hot+warm and cold on different volumes, try to maximize the use of the space." (http://wiki.splunk.com/Deploy:BucketRotationAndRetention)

0 Karma
Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...