Splunk IT Service Intelligence

Monitoring Disk Space in ITSI and having issue with Maintenance Windows

proyleJDS
Explorer

Hi

I have a inherited a KPI that monitors disk space in ITSI, the search works fine and returns a results when the thresholds are breached however the episodes continue even when the server is in maintenance mode.

I think I know why but don't yet know how to work around it.

This is the KPI search:

| mstats avg(LogicalDisk.%_Free_Space) as "logicaldisk_free_space" avg(PhysicalDisk.%_Disk_Read_Time) as "physicaldisk_read_time" avg(PhysicalDisk.%_Disk_Write_Time) as "physicaldisk_write_time" avg(Network_Interface.Packets_Received/sec) as "network_packets_received_per_second" avg(Network_Interface.Packets_Sent/sec) as "network_packets_sent_per_second" avg(Network_Interface.Bytes_Received/sec) as "network_bytes_received_per_second" avg(Network_Interface.Bytes_Sent/sec) as "network_bytes_sent_per_second" avg(Network_Interface.Packets_Outbound_Errors) as "network_packets_outbound_errors" WHERE `sai_metrics_indexes` AND instance!=_Total instance!=P: by host,instance span=30s
| eval host_dev=host . ":" . instance
| eval "physicaldisk_total_time" = physicaldisk_read_time + physicaldisk_write_time
| eval "network_packets_total_per_second" = network_packets_received_per_second + network_packets_sent_per_second
| eval "network_mbs_total_per_second" = (network_bytes_received_per_second + network_bytes_sent_per_second)/1000000

 

The Threshold field is logical_free_space

The Split by field is host_dev which as you can see combines the host name with the disk device like this
HOST1234:C:

The data is filtered by service with the host field

The result in the service analyser looks good
image (1).png

Problem is with the Entity Name now being HOSTNAME:C: when the HOST is put into maintenance this KPI keeps creating episodes.

Can someone help me with a practical way to do this and still use maintenance mode successfully?

Cheers

Labels (2)
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...