Splunk Search

Splunk Search Head : "Search not executed: The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch"

ramaswamy
New Member

From Splunk Web, when I run a search, I receive the following message

Search not executed: The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch

In Settings->ServerSettings->GeneralSettings, I have the Splunk Search Head host configured.

Splunk Search Head is having a total disk capacity of 30GB and currently 25GB is used up.

My setup has
1) One master node
2) One Search head
3) Three Peer nodes

Does the above error mean that, I am trying to index the logs of Search Head host and hence I am running short of disk space?

/volr/splunk/defaultdb/db/ seems to use 12GB of space and I believe that it is the index data.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Your disk has only 5GB free (or less) and Splunk requires at least 5GB free (5000 MB). That is why searches are not executed. To resume searching, you must release some disk space. Look for old jobs in the dispatch directory that haven't been cleaned up.

---
If this reply helps you, Karma would be appreciated.
0 Karma

ramaswamy
New Member

Does Splunk SearchHead index data? In Webui, I see a message that indexing is stopped. Also, in Settings - ServerSettings - GeneralSettings, I see Index option.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Search heads can index data and yours appears to do so. That's fine for summary indexes, but raw data should be on your indexer(s). Double-check your input settings and your forwarders to make sure data is going to the right place.

---
If this reply helps you, Karma would be appreciated.
0 Karma

vasanthmss
Motivator

richgalloway is correct. go to your server and check the disk mounted for Splunk. you can use
df -ah to validate in the unix/linux environment.

Check the below answer to clean up the dispatch,

https://answers.splunk.com/answers/139924/where-can-i-find-documentation-for-splunkd-clean-dispatch-...

Try to use SOS / Splunk Health Overview App from Splunk base. to know further about your environment.

V
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...