Monitoring Splunk

Splunk Disk Reporting Incorrect

trevor_dunstan8
Explorer

Hi all,

I am running a Splunk 7.3.0 distributed / clustered environment and I have noticed that the DMC is reporting that disk usage on my indexers is high ie around the 85% mark, however Windows Server 2016 says that it is around 65% as per attached screenshot. Its mainly the F drive as far as I can tell.

Disk usage was high previously however I then implemented retention policies on the indexes which cleared out a large amount of data, is it possible the DMC is caching an old value and is not updating ? I have restarted the Cluster master node and we have rebooted the index cluster since then I believe.

 

Any info would be great,

 

Thanks 

Labels (2)
0 Karma

trevor_dunstan8
Explorer

I thought that the Indexing cluster had been restarted but obviously not, storage figures reported correctly in DMC after cluster restart

0 Karma

isoutamo
SplunkTrust
SplunkTrust
At lest in linux version you must restart splunk after you have added/increased FS/LVM/Disks. Without restart REST queries didn't show those correctly.
r. Ismo
0 Karma

trevor_dunstan8
Explorer

We haven't extended the disks on the servers so that solution doesn't apply to us.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Defend at Machine Speed: Your Guide to Security Sessions at .conf26

Splunk .conf26   With threats moving at machine speed and attack surfaces expanding across hybrid ...

Where Innovation Takes Flight: The Splunk4Aviation Flight Sim Lands at .conf26

If you hear someone at .conf26 shouting "gear down, GEAR DOWN" across the show floor, you have found us.  The ...

Turn Cisco Telemetry Into Action with Cisco Data Fabric, powered by the Splunk ...

The surge in machine data is already hitting enterprise budgets, and the agentic era will only intensify it. ...