Tested with success, but looking for validation to ensure that this is an appropriate way to move an index to new LUN

[paimonsoror]

Hi Folks;

As our network indexes has grown rapidly over time, I am looking to preserve data and splunk performance, while making sure that we have the capacity to store the network data. In doing so, I have requested a second LUN for our network index. I have performed the following steps in my non-Prod environment, and it seems like everything was successful, but I do want to make sure that I didn't miss a step:

1. Set maintenance mode on the cluster
2. For each individual indexer
   • Stop indexer
   • edit etc/splunk-launch.conf to add a new 'SPLUNK_NETWORK_DB' variable
   • edit etc/slave-apps/all_indexes/local/indexes.conf to update the network db/thaweddb/colddb reference to use new var
   • mv var/lib/splunk/network/*db /opt/splunk_network_data
   • start indexer
3. disable maintenance mode
4. update master index file
5. deploy master index.conf to cluster to make sure all indexers are in sync

[mbuehler_splunk · Accepted solution]

Paimonsoror,

This would work, there are a few things to consider:

First, adding a new "SPLUNK_NETWORK_DB" variable is not needed, and might someday cause issues with maintainability.

I would, following best practice just change the path in the indexes.conf, that way you don't have to edit multiple files to make a "simple" change.

Second, just a word of caution, editing the Slave-apps contents can lead you down a dangerous path, so just be careful.

But yes this will work.

[paimonsoror]

Thanks for the quick response. And after thinking about it, I agree that the extra Var isn't needed. Especially because that means now if i stand up a new indexer, i need to remember to add that var to the conf file.

Regarding your second point, would there be a better alternative so that I can make sure that the indexer points to the right place for the network data when i start it back up, but before i push out a new bundle?

[mbuehler_splunk]

Painmonsoror,

I don't know that in a clustered environment you have a better option, so I would do that. because Slave-apps takes the highest precedent. So I would do it how you suggest.

Good luck!

[paimonsoror]

I appreciate it! Our nonprod testing went well, so crossing my fingers for Prod :D. Thanks again for your help