How to select a partition in the DMC Alert - Near Critical Disk Usage?

[btmoises]

Hello

I'm working on the setup of the alert when the disk space usage reaches above 80.

However, I don't how to change in the query the partition that we need to check.

The main partition is installed the Splunk service, however, i want to set the alert for another partition, the one that stores the logs.

Here is the search for the alarm:

| rest splunk_server_group=dmc_group_* /services/server/status/partitions-space
| eval free = if(isnotnull(available), available, free)
| eval usage = capacity - free
| eval pct_usage = floor(usage / capacity * 100)
| where pct_usage > 30
| stats first(fs_type) as fs_type first(capacity) AS capacity first(usage) AS usage first(pct_usage) AS pct_usage by splunk_server, mount_point
| eval usage = round(usage / 1024, 2)
| eval capacity = round(capacity / 1024, 2)
| rename splunk_server AS Instance mount_point as "Mount Point", fs_type as "File System Type", usage as "Usage (GB)", capacity as "Capacity (GB)", pct_usage as "Usage (%)"

And the result for the search:

And the partition that we need to monitoring is the next one:

Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg00-root 1014M 84M 931M 9% /
/dev/mapper/vg00-usr 4.0G 1.8G 2.3G 45% /usr
/dev/sda1 1014M 192M 823M 19% /boot
/dev/mapper/vg00-opt 10G 6.3G 3.8G 63% /opt
/dev/mapper/vg01-splunk 32G 15G 18G 47% /var/log/splunk

How can I change the query, so the search is done on the last partition?

Regards!

[gcusello]

Hi @btmoises,

this is a Linux server, so probably you installed on this server also the Splunk_TA-nix.

In this case, you already have the information about all the partitions of your system and you could run a search like the following:

index=os  sourcetype=df host=$host$ 
| head 1
| multikv 
| table Filesystem Type Size Used Avail UsePct MountedOn

to display the status of all partitions, then you can create your alert for the partition you want:

index=os  sourcetype=df host=$host$ 
| head 1
| multikv 
| table Filesystem Type Size Used Avail UsePct MountedOn
| search FileSystem="/var/log/splunk" UsePct="80%"

Ciao.

Giuseppe

[btmoises]

Hello @gcusello 

I tried the search that you suggested but I didn't get the results

 I did something wrong?

Thank you!

[gcusello]

Hi @btmoises,

$host$ is the value of a dropdown list to choose the host to monitor, you have to use the hostname of the host to monitor:

index=os  sourcetype=df host=<your_host_to_monitor> 
| head 1
| multikv 
| table Filesystem Type Size Used Avail UsePct MountedOn

Ciao.

Giuseppe

[btmoises]

Hello 

It seems that the index=os shows no data, the same with the host and sourcetype.

I use index="_*" to check all the data sources, above you can see all the sources.

So, the /dev/mapper/vg01-splunk is the $host$ right?

Thank you

[gcusello]

Hi @btmoises,

if you installed the Splunk_TA-nix, see where you archive data, usually index=os, so use this index in the search.

If you haven't this TA, install it or find a different way to extract the disk space value.

Ciao.

Giuseppe