Move Splunk's VAR folder ($SPLUNK_HOME/var or /opt/splunk/var)

[BP9906]

I've had Splunk for sever major releases (4.x to present) and now as our environment has evolved, our /opt/splunk/var/* path has many disk writes. Since Splunk v4, I've always moved SPLUNK_DB (ie /opt/splunk/var/lib/splunk) to a separate partition for indexers so that I can dedicate my fast disks (RAID 10) to indexing appropriately.

Now, I see many writes on /opt/splunk/var (yet my SPLUNK_DB resides elsewhere). I've got indexer clustering on some servers and search head clustering on others, and they all show the same behavior. I suspect its bundle replication /opt/splunk/var/run/ because I get an occasional warning that the configuration initialization took a little longer than normal.

How can I get the disk writes out of /opt/splunk/var and into my other drive with raid10 without having to move the entire splunk home folder?

[harry2007gsp]

Hi @woodcock, I tried your method and it created the soft link from external network drive to this directory /opt/splunk/val

But I can see data is still stored on the local storage rather than on network storage.

[woodcock]

Follow all the steps. Make sure that you do each one exactly. If you think that it didn't work, post the output of df.

[woodcock · Accepted solution]

The simplest way is to shut down Splunk, create a new directory on your other drive whereever you like and named whatever you like, such as mkdir /mnt/otherdrive/OptSplunkVarRun, move all the files with mv /opt/splunk/var/run/* /mnt/otherdrive/OptSplunkVarRun/, remove the old directory with rmdir /opt/splunk/var/run then create a soft link with ln -fs /mnt/otherdrive/OptSplunkVarRun /opt/splunk/var/run, and finally restart Splunk. I had to do this with the dispatch directory and it worked fine.