topic Re: How do I count the number of the occurrences / buckets when the given event happened? in Alerting

How do I count the number of the occurrences / buckets when the given event happened?

damucka — Mon, 01 Oct 2018 15:02:40 GMT

Hello,

I am trying to count the time buckets when the specific search returns values and alert on it. My current search looks as follows:

index=mlbso sourcetype=BWP_hanatraces "Out of memory for Pool/JoinEvaluator" | timechart count span=1m as OOM_Pool | eval Occurence = if (OOM_Pool > 0,1,0)

For alerting, I am only interested in the occurrence being 1 or 0, not in the number of events (count) per time bucket. Then, I want to alert when the Occurrences increase with time, which I set in the alert trigger options (> 5 in the last 30 minutes).

The problem is that this is not working and the alert takes not only the Occurrence but also a Count and adds both up. So if I have the 25 events (OOM_Pool) in one minute, then the Occurrence is 1, Count 25 and the alert gets triggered. I tried to overcome this by setting the Custom triggering condition condition:

search Occurence > 5"

but this does not seem to work.

How would I do it properly?

Kind regards,
Kamil

Re: How do I count the number of the occurrences / buckets when the given event happened?

darrenfuller — Tue, 29 Sep 2020 21:25:03 GMT

Hi @damucka

try this:

 index=mlbso sourcetype=BWP_hanatraces "Out of memory for Pool/JoinEvaluator" earliest=-30m
| bin _time span=1m 
| stats count AS OOM_Pool by _time
| eval occurrence=if(OOM_Pool!=0, 1, 0)
| stats count as occurrences_in_last_30_min

Then alert if occurrences_in_last_30_min is >5

Re: How do I count the number of the occurrences / buckets when the given event happened?

mstjohn_splunk — Mon, 01 Oct 2018 21:33:53 GMT

hi @damucka

Did the answer below solve your problem? If so, please resolve this post by approving it!
If your problem is still not solved, keep us updated so that someone else can help ya. Thanks for posting!