Monitoring Splunk

Why is Distributed Management Console (DMC) not displaying accurate DISK usage values?

nawazns5038
Builder

Hi,

I see that DMC is unable to give the right volume usage for a particular partition in the servers. It is showing the wrong partition value in every instance for that particular partition name.

Is there any specific permission that needs to be checked or how can this be fixed ?

   | rest splunk_server_group=dmc_group_* /services/server/status/partitions-space 

The above query results in false values than that of df -h in the server

Thanks

splunkoptimus
Path Finder

Facing the same problem in version 8.2.4, disk usage on the frontend is 92% and on the backend its 86%.

0 Karma

pellegrini
Path Finder

Same issue in version 8.0.3 and Red Hat Enterprise Linux Server release 7.9 (Maipo).

Looks like some customers have this issue but not all.

The free/available figures in the REST call are incorrect but the capacity is correct.

| rest splunk_server=* /services/server/status/partitions-space

 

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @nawazns5038,

| rest splunk_server_group=dmc_group_* /services/server/status/partitions-space provide values in MB, if you run df -BM the values are similar. If you want to convert those values to GB then you can use below query.

| rest splunk_server_group=dmc_group_* /services/server/status/partitions-space | eval available=round(available/1024,2), capacity=round(capacity/1024,2), free=round(free/1024,2)

I hope this helps.

Thanks,
Harshil

0 Karma

nawazns5038
Builder

Hi @harsmarvania57 ,

The DMC alert for the CRITICAL Disk usage uses the same query and calculation,

| rest splunk_server_group=dmc_group_* /services/server/status/partitions-space 
| eval free = if(isnotnull(available), available, free) 
| eval usage = capacity - free 
| eval pct_usage = floor(usage / capacity * 100) 
| where pct_usage > 80 
| stats first(fs_type) as fs_type first(capacity) AS capacity first(usage) AS usage first(pct_usage) AS pct_usage by splunk_server, mount_point 
| eval usage = round(usage / 1024, 2) 
| eval capacity = round(capacity / 1024, 2) 
| rename splunk_server AS Instance mount_point as "Mount Point", fs_type as "File System Type", usage as "Usage (GB)", capacity as "Capacity (GB)", pct_usage as "Usage (%)"

I see almost a terabyte of difference in the values and even if the backend space in the partition gets changed, the query isn't showing the updated values.

Thanks

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Which version of splunk are you running so that I will try to replicate this issue.

0 Karma

nawazns5038
Builder

version 6.5.3

0 Karma

thambisetty
SplunkTrust
SplunkTrust

facing same issue in 6.6.4 also.

————————————
If this helps, give a like below.
0 Karma

nawazns5038
Builder

Did you raise a support case ?
They have suggested me to upgrade Splunk to 6.6+

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...