Monitoring Splunk

Why is Distributed Management Console (DMC) not displaying accurate DISK usage values?

nawazns5038
Builder

Hi,

I see that DMC is unable to give the right volume usage for a particular partition in the servers. It is showing the wrong partition value in every instance for that particular partition name.

Is there any specific permission that needs to be checked or how can this be fixed ?

   | rest splunk_server_group=dmc_group_* /services/server/status/partitions-space 

The above query results in false values than that of df -h in the server

Thanks

splunkoptimus
Explorer

Facing the same problem in version 8.2.4, disk usage on the frontend is 92% and on the backend its 86%.

0 Karma

pellegrini
Path Finder

Same issue in version 8.0.3 and Red Hat Enterprise Linux Server release 7.9 (Maipo).

Looks like some customers have this issue but not all.

The free/available figures in the REST call are incorrect but the capacity is correct.

| rest splunk_server=* /services/server/status/partitions-space

 

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @nawazns5038,

| rest splunk_server_group=dmc_group_* /services/server/status/partitions-space provide values in MB, if you run df -BM the values are similar. If you want to convert those values to GB then you can use below query.

| rest splunk_server_group=dmc_group_* /services/server/status/partitions-space | eval available=round(available/1024,2), capacity=round(capacity/1024,2), free=round(free/1024,2)

I hope this helps.

Thanks,
Harshil

0 Karma

nawazns5038
Builder

Hi @harsmarvania57 ,

The DMC alert for the CRITICAL Disk usage uses the same query and calculation,

| rest splunk_server_group=dmc_group_* /services/server/status/partitions-space 
| eval free = if(isnotnull(available), available, free) 
| eval usage = capacity - free 
| eval pct_usage = floor(usage / capacity * 100) 
| where pct_usage > 80 
| stats first(fs_type) as fs_type first(capacity) AS capacity first(usage) AS usage first(pct_usage) AS pct_usage by splunk_server, mount_point 
| eval usage = round(usage / 1024, 2) 
| eval capacity = round(capacity / 1024, 2) 
| rename splunk_server AS Instance mount_point as "Mount Point", fs_type as "File System Type", usage as "Usage (GB)", capacity as "Capacity (GB)", pct_usage as "Usage (%)"

I see almost a terabyte of difference in the values and even if the backend space in the partition gets changed, the query isn't showing the updated values.

Thanks

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Which version of splunk are you running so that I will try to replicate this issue.

0 Karma

nawazns5038
Builder

version 6.5.3

0 Karma

thambisetty
Super Champion

facing same issue in 6.6.4 also.

————————————
If this helps, give a like below.
0 Karma

nawazns5038
Builder

Did you raise a support case ?
They have suggested me to upgrade Splunk to 6.6+

0 Karma
Get Updates on the Splunk Community!

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...

Security Highlights | January 2023 Newsletter

January 2023 Splunk Security Essentials (SSE) 3.7.0 ReleaseThe free Splunk Security Essentials (SSE) 3.7.0 app ...