topic Re: Disk Space Error in Monitoring Splunk

Disk Space Error

harshavrath — Tue, 11 Mar 2014 09:38:52 GMT

I'm getting this error

You are low in disk space on partition C:\Program Files\Splunk\var\lib\splunk\audit\db. Indexing has been paused. Will resume when free disk space rises above 5000MB.

How can i change the drive as can not increase the space in my C Drive.I have space in my other drives.

Any Suggestions are Appreciated,

Cheers.

Re: Disk Space Error

MuS — Tue, 11 Mar 2014 09:54:51 GMT

Hi harshavrath,

well, you have installed Splunk on the C: drive and you're using the C: drive as well for your indexes. Read about the indexes.conf how to change the homePath for your indexes.

To move your existing data to another location proceed like this:

stop Splunk
change the homePath in indexes.conf
move all existing data from old homePath (in your case C:\Program Files\Splunk\var\lib\splunk) to new homePath
start Splunk

hope this helps ...

cheers, MuS

Re: Disk Space Error

harshavrath — Tue, 11 Mar 2014 12:12:58 GMT

Hi MuS

Can i directly change the path to f drive instead of c drive under the Splunk UI
Settings>System Settings>General Settings>Index Settings(Path to Indexes)

Re: Disk Space Error

MuS — Tue, 11 Mar 2014 12:23:23 GMT

Yes, this will change the setting globally for all indexes. But you still have to move the files from 'old path' to 'new path' while Splunk is not running.

Re: Disk Space Error

harshavrath — Tue, 11 Mar 2014 13:15:38 GMT

if i stop splunk how will i use the UI.?
As i require the UI to specify the new path under
Settings>System Settings>General Settings>Index Settings(Path to Indexes)

Re: Disk Space Error

MuS — Tue, 11 Mar 2014 13:21:52 GMT

okay, you can restart Splunk from the UI as well since you don't want to keep the old data. Just in case you want/need to keep the old indexes, change the setting in the UI but don't restart yet. do a 'net stop splunkd', move data and do a 'net start'. You can stop/start Splunkweb, but this is not mandatory for this action.

Re: Disk Space Error

harshavrath — Tue, 11 Mar 2014 13:45:01 GMT

Thanks this worked now its showing Events Indexed:163,153

Re: Disk Space Error

harshavrath — Tue, 11 Mar 2014 13:51:37 GMT

I'm getting a new Error
The minimum free disk space (5000MB) reached for
(C:\Program Files\Splunk\var\run\splunk\dispatch)

Re: Disk Space Error

MuS — Tue, 11 Mar 2014 13:57:00 GMT

seams your C: drive is pretty tight on space, try to clean up to gain disk space and lower the disk space alert for splunk in server.conf minFreeSpace option or in the UI > settings > general settings > 'Pause indexing if disk space ...'

Re: Disk Space Error

harshavrath — Tue, 11 Mar 2014 14:02:15 GMT

In my C drive i have only 900MB so under general settings how much space do you advice me to set for "Pause Indexing.."

Re: Disk Space Error

MuS — Tue, 11 Mar 2014 14:06:05 GMT

look, since this looks like a newer install with almost no historical data I would suggest that you uninstall Splunk from your C: drive and install Splunk on a drive which holds more free space.

Re: Disk Space Error

harshavrath — Tue, 11 Mar 2014 14:16:11 GMT

I can't do that,instead can i change in General Settings for "Pause Indexing" to 500MB

Re: Disk Space Error

MuS — Tue, 11 Mar 2014 14:18:24 GMT

sure, but you will hit the same problem again and again as soon as there is some search activity going on, because this is were your search results/artifacts are stored....

Re: Disk Space Error

harshavrath — Tue, 11 Mar 2014 14:23:19 GMT

How can i clear/delete the indexed data.?

Re: Disk Space Error

MuS — Tue, 11 Mar 2014 14:26:55 GMT

read this http://docs.splunk.com/Documentation/Splunk/6.0.2/Indexer/RemovedatafromSplunk

but this will have no effect on your disk space problem on the C: drive, because you moved the data in the indexes to another drive

Re: Disk Space Error

harshavrath — Tue, 11 Mar 2014 14:29:59 GMT

Thanks For Your Valuable Info.

Re: Disk Space Error

dmaislin_splunk — Wed, 30 Apr 2014 12:21:05 GMT

Stop Splunk. Erase all your logs under $SPLUNK_HOME/var/log/splunk
Open Command Line and cd to the $SPLUNK_HOME/bin directory.
Type splunk clean eventdata
Enter your splunk admin / password
Start Splunk

Re: Disk Space Error

nawazns5038 — Wed, 06 Dec 2017 19:28:07 GMT

how to do it in a cluster .... how to stop and move .... Can we delete the _internal data instead and change the homepath so that the new data goes to the new path