Why is Distributed Management Console (DMC) not displaying accurate DISK usage values?

[nawazns5038]

Hi,

I see that DMC is unable to give the right volume usage for a particular partition in the servers. It is showing the wrong partition value in every instance for that particular partition name.

Is there any specific permission that needs to be checked or how can this be fixed ?

   | rest splunk_server_group=dmc_group_* /services/server/status/partitions-space 

The above query results in false values than that of df -h in the server

Thanks

[splunkoptimus]

Facing the same problem in version 8.2.4, disk usage on the frontend is 92% and on the backend its 86%.

[pellegrini]

Same issue in version 8.0.3 and Red Hat Enterprise Linux Server release 7.9 (Maipo).

Looks like some customers have this issue but not all.

The free/available figures in the REST call are incorrect but the capacity is correct.

| rest splunk_server=* /services/server/status/partitions-space

[harsmarvania57]

Hi @nawazns5038,

| rest splunk_server_group=dmc_group_* /services/server/status/partitions-space provide values in MB, if you run df -BM the values are similar. If you want to convert those values to GB then you can use below query.

| rest splunk_server_group=dmc_group_* /services/server/status/partitions-space | eval available=round(available/1024,2), capacity=round(capacity/1024,2), free=round(free/1024,2)

I hope this helps.

Thanks,
Harshil

[nawazns5038]

Hi @harsmarvania57 ,

The DMC alert for the CRITICAL Disk usage uses the same query and calculation,

| rest splunk_server_group=dmc_group_* /services/server/status/partitions-space 
| eval free = if(isnotnull(available), available, free) 
| eval usage = capacity - free 
| eval pct_usage = floor(usage / capacity * 100) 
| where pct_usage > 80 
| stats first(fs_type) as fs_type first(capacity) AS capacity first(usage) AS usage first(pct_usage) AS pct_usage by splunk_server, mount_point 
| eval usage = round(usage / 1024, 2) 
| eval capacity = round(capacity / 1024, 2) 
| rename splunk_server AS Instance mount_point as "Mount Point", fs_type as "File System Type", usage as "Usage (GB)", capacity as "Capacity (GB)", pct_usage as "Usage (%)"

I see almost a terabyte of difference in the values and even if the backend space in the partition gets changed, the query isn't showing the updated values.

Thanks

[harsmarvania57]

Which version of splunk are you running so that I will try to replicate this issue.

[nawazns5038]

version 6.5.3

[thambisetty]

facing same issue in 6.6.4 also.

[nawazns5038]

Did you raise a support case ?
They have suggested me to upgrade Splunk to 6.6+