Splunk Enterprise

Indexes.conf Storage Configuration

jlc00
Loves-to-Learn Lots

I am in the middle of configuring a standalone Splunk installation.  I am getting confused about the different attributes that can be set for overall storage and per index.  It is a very small installation with only about 30 assets connected and about 2.3TB of storage to store data for a year.  I have the following configuration so far:

frozenTimePeriodinSecs = 31536000 #365 days

[volume:hotwarm]
path = <directory to hotwarm location>
maxVolumeDataSizeMB = 178176 #174GB

[volume:cold]
path = <directory to cold location>
maxVolumeDataSizeMB = 1970176 #1924GB

[network]
homePath = volume:hotwarm/network/db
coldPath = volume:cold/network/colddb
thawedPath = $SPLUNK_DB/network/thaweddb

[windows]
homePath = volume:hotwarm/windows/db
coldPath = volume:cold/windows/colddb
thawedPath = $SPLUNK_DB/windows/thaweddb

I'm not sure how to use the maxTotalDataSizeMB with maxVolumeDataSizeMB to keep from maxTotalDataSizeMB triggering a roll to frozen before the 365 days is up.  We currently do not have any idea how much data will be coming in.  Is it good practice to set maxTotalDataSizeMB for each index to the same size as maxVolumeDataSizeMB? I have seen this practice before...   And if so, is it the maxVolumeDataSize of the cold storage, or hot/warm/cold storage combined?

Labels (2)
0 Karma

Stefanie
Builder

I think you'll need to monitor your data for at least a month to see how much you're getting in, in order to fine tune your settings. 

Personally, I wouldn't set maxTotalDataSizeMB for each index to equal the same size as the maxVolumeDataSizeMB. If it's set the same, what happens in the event one of your indexes fills up that maxVolumeDataSizeMB faster than your other indexes? It won't leave enough room for your other indexes.

I think setting only the maxVolumeDataSizeMB will be fine until you monitor your ingestion to see which indexes are your busiest, and fine tune your settings from there.

 

 

 

richgalloway
SplunkTrust
SplunkTrust

maxTotalDataSizeMB sets the maximum size of an individual index whereas maxVolumeDataSizeMB sets the maximum size of all indexes sharing that volume.

The maximum size of an index applies to hot/warm as well as cold.  However, if hot/warm and cold are on different volumes then the max index size should not exceed the larger volume size.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...