Splunk Enterprise

Can I change the SPLUNK_DB value of my indexers inside my cluster?

LinghGroove
Explorer

Hello, I would like to change the SPLUNK_DB value of my indexers inside my cluster. I want to write the logs to another directory but i don't want to bring the old logs. Will this cause problems? Could you please provide the detailed steps on how to do it? I have already tried to change the path on splunk-launch.conf (and the environment variable value on the os) but the servers keep writing in the old directory.  

Thanks a lot

0 Karma
1 Solution

isoutamo
SplunkTrust
SplunkTrust

You should check how those are defined on /opt/splunk/etc/master-apps/_cluster (could be manager-apps also) or where ever you have defined those on your cluster master.

You could check the place where those are defined on your peer by

splunk btool indexes list <your index name> --debug | egrep -i path

That shows how those are define on peer side and where that indexes.conf is located. Based on that information it should be quite easy to find corresponding file on manger/master node. Then just fix it if needed and apply cluster bundle again. 

View solution in original post

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

you can do it, but exact way depends how your indexes are defined. Are you using volumes or only referring to SPLUNK_DB? Based on your experience when you have changed SPLUNK_DB and nothing happened, I guess that you are using volumes. I suppose that those volumes are defined on some index.conf on your cluster master.  First you must find that and then change its definition to your new file system. Then just apply cluster bundle and your peers will do rolling restart and then it should works.  If you want to access old events, then this needs some other steps before you can do apply and later some more.

r. Ismo

LinghGroove
Explorer

Hello,

thanks a lot for your response. I'm sorry if i haven't specified this first. No, I am not using volumes. This is just a test environment. At the moment all the indexes are configured with the $SPLUNK_DB variable. When i tried to change the value of the variable and after the restart Splunk created the folders for the indexes in the new directory but it started writing again buckets on the default folder. This accurres even for the _* indexes.  

0 Karma

isoutamo
SplunkTrust
SplunkTrust

You should check how those are defined on /opt/splunk/etc/master-apps/_cluster (could be manager-apps also) or where ever you have defined those on your cluster master.

You could check the place where those are defined on your peer by

splunk btool indexes list <your index name> --debug | egrep -i path

That shows how those are define on peer side and where that indexes.conf is located. Based on that information it should be quite easy to find corresponding file on manger/master node. Then just fix it if needed and apply cluster bundle again. 

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...