The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch - But I have over 5 GB Free...

[asbetsplunk]

Splunk single instance install on AWS.

Splunk Version: 6.3.0
Splunk Build: aa7d4b1ccb80

I am getting the error message, "The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch" however I have over 5 GB on my base install.

Also, had previously already moved all of my index data to a different volume before this error appeared.

See screenshots below - why am I getting this error and what do I need to do?

I'm assuming that I need to move the dispatch to my separate index volume but not understanding why I'm getting the error in the first place.

[vasanthmss]

Hi there,

Splunk required 5 GB in the required mount. with your configuration you installed in the tiny mount /dev/xvdal. Why don't you mount combine the same with /dev/xvdf. because the mount you were refereeing has lot of space. Is there any reason you were not installed in the /splunk_index_volume mount?

thanks,
V

[asbetsplunk]

Thanks @vasanthmss for your reply.

I'm not sure I understand - yes, the mount is small but it is larger than 5 GB so why is the error generating? If the free space /dev/xvda1 was 4.9 GB or exactly 5 GB then I completely understand.

The reason why the first partition is small is because the AWS Enterprise instance offered in the AWS Marketplace defaults to 8 GB. I kept that size and added another volume for index data because well, obviously 8 GB minus a Splunk install isn't going to be enough for anything.

Unfortunately, I didn't know that there is another file (i.e. ./dispatch) that grows in size as well (I'm assuming - still have minimum free space so don't get it) - that is why I didn't know that I needed to move it.

For now I reduced the minimum free disk space threshold to 4 GB so I can at least search.

1. Is there a way to safely move the dispatch file using the command line? I'm asking because I see references to symbolic links that some say worked and others didn't. https://answers.splunk.com/answers/2205/can-i-change-the-path-of-the-dispatch-directory.html#answer-...
2. Are there any other "gotchas" like this that we should be aware of?