AppD Archive

Use time period or average value in JVM Heap Utilization alert

Pavel_Kouzmitch
Explorer

I set the JVM heap utilization alert very similar to the default one. It sends alert when the JVM Heap Utilization reaches 96%.

The problem is that normally JVM very often reaches 96% and then Garbage Collection runs and cleans the memory normally. Hence, as there are multple instances in my system, I receive a lot of alerts every hour, and these alerts are sent when the system isn't experiensing any problems.

Is it possible to set the health rule like: "Trigger Alert When JVM Heap Utilization remails on > XX% for Y seconds" ?

In other words, is it possible to send the alert *not* right after JVM Heap Utilization reaches XX%, but instead, if this high utilization remains for some period of time? Like 3 minutes for example. This will definitelly tell me that there is some issue, and GC is not able to release the memory, and there is a real issue with the system.

Other alternative is probably to use Average Usage % for some period, but I didn't find this option in the Condition dropdown. The only options are: Minumum, Maximum, Value, Sum, Count, Current. Is there any chance to add "Average" to this options, and it will mean to calculate average value for the period mentioned in Overview part of the rule (30 minutes by default), or use a separate value setting to identify the period for the average.

See below the screen shots of the Rule I'm talking about image.jpegimage.jpeg

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hi,

I dont think there is an easy way to do this without additional conditions and better logic.

In your case of JVM monitoring, look into your JVM GC stats  or any other  JVM/hardware stat that you could get to corelate the heap usage.

0 Karma

Pavel_Kouzmitch
Explorer

Thanks dhiren.rana, this is good idea for the workaround.

I created a health rule sending the alert in case there were more than 1 Major Garbage Collections in a minute.

Hopefully this will work!

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...