We are running out of space for Hot/Warm data, so as a short term work around I am trying to get splunk to log HotWarm data under the colddb disk as we have lots of disk space there.
dev1 /opt/splunk/var/ <<<< running out of space (This is where HotWarm goes)
dev2 /opt/splunk/colddb/ <<<<< lots and lots of space (This is where cold data goes)
created new location /opt/splunk/colddb/splunkdb (owned by the splunk user, for the Hot/Warm data)
I have stopped splunk, recursively copied over all the indexes (preserving permissions) to the new location, pointed SPLUNK_DB to it in splunk-launch.conf
So we should use the device diskspace for cold data for hot/warm too under the splunkdb/
...../local/indexes.conf uses the $SPLUNK_DB variable for the homePath's
I then restarted splunk, but, $ echo $SPLUNK_DB still shows as /opt/splunk/var/lib/splunk and data of course still goes there.
So my setting under splunk-launch.conf is not working.
Further to that, we have splunk installed under a splunk user, under that users home directory is the .bash_profile, I can force SPLUNK_DB under there,
echo $SPLUNK_DB then shows the correct path, YET! after restarting splunk, hot/warm data still logs to the default /opt/splunk/var/lib/splunk/
Does anyone know why the setting in splunk-launch.conf would be overridden?
And why the .bash_profile setting doesnt work either?
I would use btool, but not sure how to for this problem.
OK it appears to be working, I am getting disk space used up, but its not the index data. the index data is behaving as it should and now going to the $SPLUNK_DB as specified in splunk-launch.conf, my bad.
It looks like I have spool data taking up space, so I probably have a different issue to look at.