Splunk Search

Search results in dispatch command message "The minimum free disk space (18446744073709551615MB) reached"

dccrain
New Member

Recently I migrated one of our indexers to a new machine.

Sometimes searches result in the below message despite there being plenty of space in the /opt/splunk mount and the minFreeSpace being set to 2000 in /opt/splunk/etc/system/local/server.conf.

Dispatch Command: The minimum free disk space (18446744073709551615MB) reached for /opt/splunk/var/run/splunk/dispatch dispite [diskUsage]minFreeSpace = 2000
0 Karma

dccrain
New Member

Found the problem: the /opt/splunk/var/run/splunk/dispatch dir was a symlink to a different storage array on the old machine, and was broken when rsync-ing from the the old host to the replacement, which has a single large /opt/ mount for splunk. Deleting the broken symlink and making a proper dir and then copying data again from the original host cleared up the errors.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

run a btool to check where is that config coming from. still its weird since you have the config in system/local

/opt/splunk/bin/splunk btool server list --debug disk

Also which Splunk version are you using?

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

nick405060
Motivator

Well 18446744073 petabytes is a lot for just one indexer. I'd spread it out across... two indexers

0 Karma
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...