topic Re: How to resolve this error: The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch. use in Monitoring Splunk

How to resolve this error: "The minimum free disk space (5000MB)"

jaracan — Wed, 11 May 2022 22:35:43 GMT

Hi,

I had this error below:
search:262 - reason="Search not executed: Dispatch Manager: The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch. user=admin.", concurrency_category="historical", concurrency_context="user_instance-wide", current_concurrency=0, concurrency_limit=5000

I do not want to change the limit 5000MB.
I already had deleted some logs in /opt/splunk/var/log/splunk like metrics.log.* but it did not resolve the issue.
Also try the clean-dispatch command, and also deleted the files inside the /opt/splunk/var/run/splunk/dispatch.

Is there any suggestion you could share with me excluding the change the set limit of 5000MB?

Re: How to resolve this error: The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch. user=admin", concurrency_category="historical", concurrency_context="user_instance-wide", current_concurrency=0, concurrency_limit=5000

p_gurav — Fri, 20 Apr 2018 08:28:51 GMT

Can you check "Indexing: Indexes and volumes" page in monitoring console to get more details?
Also you can try to adjust no of concurrent searches in limits.conf

Re: How to resolve this error: The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch. user=admin", concurrency_category="historical", concurrency_context="user_instance-wide", current_concurrency=0, concurrency_limit=5000

jaracan — Fri, 20 Apr 2018 08:36:01 GMT

Hi p_gurav,

The search capability is disabled.
Also, can you specify which parameter/attribute in limits.conf should I adjust?

Re: How to resolve this error: The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch. user=admin", concurrency_category="historical", concurrency_context="user_instance-wide", current_concurrency=0, concurrency_limit=5000

adonio — Fri, 20 Apr 2018 13:16:40 GMT

see here:

https://answers.splunk.com/answers/214804/the-minimum-free-disk-space-5000mb-reached-for-opt.html

there are more answers about this subject

Re: How to resolve this error: The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch. user=admin", concurrency_category="historical", concurrency_context="user_instance-wide", current_concurrency=0, concurrency_limit=5000

woodcock — Sun, 22 Apr 2018 22:01:35 GMT

Best answer: redo your search head so that /opt/splunk/var is in its own disk volume so that nothing else can take up this space. Some people even go so var as to put /opt/splunk/var/run/dispatch into its own volume.

The next best answer: add more disk space to to the volume that contains the dispatch directory.

OK answer: Reduce the TTL for search artifacts so things auto-purge more quickly as described here: https://www.splunk.com/blog/2012/09/12/how-long-does-my-search-live-default-search-ttl.html

The WORST answer: reduce this quota in Settings -> Server settings -> General settings -> Pause indexing if free disk space (in MB) falls below.

Re: How to resolve this error: The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch. use

dionelhani — Mon, 28 Jun 2021 14:42:25 GMT

Thanks for the answers...and the pun 😉

Re: How to resolve this error: The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch. use

codebuilder — Tue, 29 Jun 2021 17:58:25 GMT

Another option is to configure logrotate to both rotate and compress your Splunk log files, then blacklist .gz files.

Re: How to resolve this error: The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch. use

Atchyuth_P — Wed, 11 May 2022 17:12:44 GMT

Hi @jaracan

Please execute the commands in the terminal

## Fix minimum free diskspace issue - only for lab environment not for production
echo "[diskUsage]" >> /opt/splunk/etc/system/local/server.conf
echo "minFreeSpace = 50" >> /opt/splunk/etc/system/local/server.conf

After this restart the splunk service