Monitoring Splunk

Splunk Disk Reporting Incorrect

trevor_dunstan8
Explorer

Hi all,

I am running a Splunk 7.3.0 distributed / clustered environment and I have noticed that the DMC is reporting that disk usage on my indexers is high ie around the 85% mark, however Windows Server 2016 says that it is around 65% as per attached screenshot. Its mainly the F drive as far as I can tell.

Disk usage was high previously however I then implemented retention policies on the indexes which cleared out a large amount of data, is it possible the DMC is caching an old value and is not updating ? I have restarted the Cluster master node and we have rebooted the index cluster since then I believe.

 

Any info would be great,

 

Thanks 

Labels (2)
0 Karma

trevor_dunstan8
Explorer

I thought that the Indexing cluster had been restarted but obviously not, storage figures reported correctly in DMC after cluster restart

0 Karma

isoutamo
SplunkTrust
SplunkTrust
At lest in linux version you must restart splunk after you have added/increased FS/LVM/Disks. Without restart REST queries didn't show those correctly.
r. Ismo
0 Karma

trevor_dunstan8
Explorer

We haven't extended the disks on the servers so that solution doesn't apply to us.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Self-Healing Pipeline Is Now Generally Available: AI-Powered CIM Compliance

Maintaining data integrity across security and analytics pipelines is an ongoing challenge. Data ...

Meet Splunk Observability Studio: AI-Assisted OpenTelemetry Instrumentation Without ...

Instrumentation is usually the last step or even an afterthought when building out a project. The feature ...

Federated Search for Cisco Security and Analytics Logging (SAL) is now GA on Splunk ...

Federated Search for Cisco  Security Analytics and Logging (SAL) is now generally available as part of the ...