topic How to resolve disk usage show negative value on monitoring console after adding storage. in Splunk Enterprise

How to resolve disk usage show negative value on monitoring console after adding storage.

elend — Fri, 01 Nov 2024 03:52:12 GMT

Hi, i made changes on my indexer storage but when i see on monitoring console part disk usage, the value is negative. Have anyone face this?. I already refresh the asset with monitoring console refresh and restart the instance but nothing changed.

Re: How to resolve disk usage show negative value on monitoring console after adding storage.

PickleRick — Fri, 01 Nov 2024 22:09:43 GMT

In which panel and which value is negative?

Anyway, you can open any panel in search and see where this value comes from. Most probably there is an initial rest call which returns wrong values but you have to double-check that.

Did you restart splunkd on the server(s) where you added storage or did you just extend the filesystem on the fly?

Re: How to resolve disk usage show negative value on monitoring console after adding storage.

elend — Sat, 02 Nov 2024 10:36:52 GMT

Its on monitoring console, Disk Usage part. If we see there, there is a bar show the storage used. I already restart the service too, but the value still like the picture. I also inspect search from the panel, and the value are same, i copy it on excel like this one.

Re: How to resolve disk usage show negative value on monitoring console after adding storage.

PickleRick — Sat, 02 Nov 2024 12:54:20 GMT

Did you check the results which the initial rest command yields?

Re: How to resolve disk usage show negative value on monitoring console after adding storage.

isoutamo — Sat, 02 Nov 2024 13:19:57 GMT

at least some older splunk versions (e.g. 7.3.x) there was (probably) bug which leads this kind of behavior when you are using REST with a increased storage areas like increased filesystem. The fix was restart splunkd.

r. Ismo

Re: How to resolve disk usage show negative value on monitoring console after adding storage.

elend — Mon, 04 Nov 2024 04:34:01 GMT

but the restart process already done and it still show same value

Re: How to resolve disk usage show negative value on monitoring console after adding storage.

elend — Tue, 05 Nov 2024 03:45:57 GMT

as i know, the result is same

Re: How to resolve disk usage show negative value on monitoring console after adding storage.

elend — Tue, 05 Nov 2024 04:37:53 GMT

this /cold part

Re: How to resolve disk usage show negative value on monitoring console after adding storage.

PickleRick — Tue, 05 Nov 2024 08:13:15 GMT

OK. This is indeed interesting.

The search behind this panel uses the /services/server/status/partitions-space REST endpoint.

This endpoint, according to docs, returns four values.

- capacity

- free

- fs_type

- mount_point

(along with some "standard" fields like title, author, id and eai stuff)

But the actual data returned by the call also includes a field called "available". And in my case the "available" field indeed shows the free space on the filesystem. The "free" field (again - in my case) contains some value completely unrelated to anything.

But the search behind the MC panel uses the field "available" if it's included in the data. If it's not included, it uses the "free" field.

Check the results of

| rest splunk_server=<your indexer> /services/server/status/partitions-space
| fields - eai* id author published updated title

And see if the data makes sense. I suspect you're not getting the "available" field and your "free" field contains some bonkers value.

EDIT: Posted a feedback to the docs page describing this REST endpoint