Getting Data In

Can anyone provide guidance on my plan to configure cold storage in indexes.conf?

ccsfdave
Builder

So, I got the 150TB cold, but they are mounted into /mnt/splunk1/cold and /mnt/splunk2/cold. I figured that may cause issues with the indexers, so I made symlinks to /opt/splunk/var/lib/splunk/cold on each of the indexers to prevent issues with which indexer Splunk wants to write to.

I am now thinking about changing the indexes.conf and adding to the volume stanza:

# One Volume for Cold
[volume:cold]
path = /opt/splunk/var/lib/splunk/cold
# 150000GB (150TB)
maxVolumeDataSizeMB = 150000000

Then changing the cold locations from:
coldPath = volume:primary/defaultdb/colddb
to
coldPath = volume:cold/defaultdb/colddb

The ES definitions are:
coldPath = $SPLUNK_DB/audit_summarydb/colddb

I would like to change that too, similar to above:
coldPath = volume:cold/audit_summarydb/colddb

Thoughts? Guidance?

0 Karma
1 Solution

ccsfdave
Builder
  1. I changed the coldPath on all my indexes to volume:cold
  2. I created a /opt/splunk/etc/system/local/indexes.conf on my SH and Indexers

maxWarmDBCount = 50
maxHotSpanSecs = 2592000

Anything else I should or shouldn't have done?

View solution in original post

0 Karma

ccsfdave
Builder
  1. I changed the coldPath on all my indexes to volume:cold
  2. I created a /opt/splunk/etc/system/local/indexes.conf on my SH and Indexers

maxWarmDBCount = 50
maxHotSpanSecs = 2592000

Anything else I should or shouldn't have done?

0 Karma

ccsfdave
Builder

Well, the above worked for me. In our case we have 675GB SSD RAID 1 each on two indexers and they were full with the default settings. I finally got the 150TB of spinning drives mounted in as cold but nothing was rolling over to it. So I did a search of my data to see how far it went back. Not sure this was scientific in anyway but I decided to 1/3 the default settings above with the end results being .

The end result was I brought my hot drives to 60% and 72% utilization. So we will go forward with this config until I get more hot drives.

Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...