Deployment Architecture

The minimum free disk space (5000MB) reached, solve by increasing disk space in Linux?

wuming79
Path Finder

Hi,

Recently I encountered the issue "The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch". My splunk is installed on a vm so I just tried to give more disk space to linux by just increasing the provision space. Somehow, the splunk message is no more. However, I read that I still need to do something to the linux partition table? Do I really need to do the change to partition table in linux to really solved the disk space issue for splunk?? https://www.unixmen.com/increase-disk-space-and-memory-in-linux-vmware-virtual-machines/

0 Karma

wuming79
Path Finder

So increasing the disk size from vsphere resulted in the "minimum free disk space (5000MB) reached" disappearing is a bug?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @wuming79,

why a bug?

it's a protection for your system: if you increase the dimension of your file system, you'll have more than 5 GB free so indexing will restart.

As I said, you could reduce Free Space limit to continue indexing and in he meantime, you have to enlarge file system, then you can set again Free space limit to 5GB.

In this way you will not have the pause in indexing.

In addition, you could do a Capacity Planning to understand how much you have to enlarge the file system.

Ciao.

Giuseppe

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi wuming79,
Splunk stops indexing when the minimum disk space is reached.
You can reduce minimum disk space limit (e.g. to 2 GB) in [Settings -- Server Settings -- General Settings -- Pause indexing if free disk space (in MB) falls below] but it's only a workaround while you solve the problem, at the end of activity set newly the limit to 5GB.

Anyway you have to give more space to your VM by VM-Ware console (or another VM console) and then enlarge your filesystem.
You can find in Internet how to do this, e.g. on Linux you have to

  • stop VM
  • delete old snapshots
  • give more space using VM console
  • snapshot of the new VM
  • restart VM
  • lvextend -LxxG LV
  • resize2fs LV

Bye.
Giuseppe

Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...