Dispatch directory full

[enzomialich]

Hi, if an environment encounters the error 'Search not executed: The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch' multiple times (meaning the issue persists even after cleaning the dispatch directory), what corrective actions should be taken? Should the dispatch directory be cleaned regularly?
This is for a standalone environment.

[enzomialich]

edited

[PickleRick · Accepted solution]

It's up to your OS and/or Splunk admins to solve. For some reason the filesystem on which the dispatch directory is located is filled up to the brim. It might be just the dispatch data but if it's on the same filesystem as - for example - Splunk's internal logs and maybe OS logs and other stuff there could be other places you need to be looking for free space.

[burwell]

Here's some good information about the dispatch directory: https://docs.splunk.com/Documentation/Splunk/9.3.1/Search/Dispatchdirectoryandsearchartifacts

Splunk normally does age things out but read the doc above.

Perhaps the disk is full for other reasons? https://community.splunk.com/t5/Splunk-Search/Splunk-says-dispatch-directory-is-full-but-when-I-go-t...

One thing that can cause your dispatch directory to grow is if you adjust the time to live (TTL) of jobs.

[enzomialich]

If I put a lower value on TTL for the dispatch directory - this would be a good idea in this case?