Alerting

When the following search sees a percentage increase, can you help me create an alarm on it?

amirarsalan
Explorer

Hi, I have a search that looks like this:

index=loadbalancer r_host="sport.mtm.com"  req="/api/v2/log/exception" 

Now, I want to create an alarm on it. I want it to alarm when it sees a percentage increase.

Can anyone help me?

Tags (2)
0 Karma
1 Solution

vishaltaneja070
Motivator

@amirarsalan
Try this:

index=loadbalancer r_host="sport.mtm.com" req="/api/v2/log/exception" earliest=-2h latest=-1h | stats count | appendcols [ search index=loadbalancer r_host="sport.mtm.com" req="/api/v2/log/exception" earliest=-1h latest=now() | stats count as count2] | eval perc= round(((count2 - count) * 100 / count),2) | fields + perc | search perc < 50

View solution in original post

vishaltaneja070
Motivator

@amirarsalan
Try this:

index=loadbalancer r_host="sport.mtm.com" req="/api/v2/log/exception" earliest=-2h latest=-1h | stats count | appendcols [ search index=loadbalancer r_host="sport.mtm.com" req="/api/v2/log/exception" earliest=-1h latest=now() | stats count as count2] | eval perc= round(((count2 - count) * 100 / count),2) | fields + perc | search perc < 50

vishaltaneja070
Motivator

Hello @amirasalan

Is there percentage field available in data?

Or you want based on no of events per minutes or hour etc?

0 Karma

amirarsalan
Explorer

Hi @vishaltaneja07011993

I want it on number on event per hour. But i only want alert when it sees a percentage increase.

0 Karma

vishaltaneja070
Motivator

Try something:

index=* earliest=-2h latest=-1h | stats count | appendcols [ search index=* earliest=-1h latest=now() | stats count as count2] | eval perc= round(((count2 - count) * 100 / count),2) | fields + perc | search perc < 0

Now you can create a alert based on the above search.

0 Karma

amirarsalan
Explorer

I don't receive any results, should it be like that? and the other question is what value should i use when i create the alert on the trigger conditions? see the link picture
https://www.google.com/search?q=create+alert+splunk&rlz=1C1GCEB_enSE814SE814&source=lnms&tbm=isch&sa...

0 Karma

amirarsalan
Explorer

I only see events not statistic

0 Karma

vishaltaneja070
Motivator
index=loadbalancer r_host="sport.mtm.com" req="/api/v2/log/exception"  earliest=-2h latest=-1h | stats count | appendcols [ search index=loadbalancer r_host="sport.mtm.com" req="/api/v2/log/exception"  earliest=-1h latest=now() | stats count as count2] | eval perc= round(((count2 - count) * 100 / count),2) | fields + perc | search perc < 0
0 Karma

vishaltaneja070
Motivator

Did you try this?

0 Karma

amirarsalan
Explorer

Now i got results -100,00
I believe its right.
How do I create alert on that

0 Karma

vishaltaneja070
Motivator

You can save the search as Alert.

0 Karma

amirarsalan
Explorer

Thanks, i'm i kind of newbee hehe. in Trigger Conditions what value should i put

0 Karma

vishaltaneja070
Motivator

You can put the condition when number of results is more than 0.

0 Karma

amirarsalan
Explorer

Okey i will do that. Last question, so the alert will trigger when its sees a big percentage increase. I forgot to mention that. I want it to trigger when it's a big percentage increase

0 Karma

vishaltaneja070
Motivator

Okay at what percentage you want alert?

0 Karma

amirarsalan
Explorer

50 % at least

0 Karma

vishaltaneja070
Motivator

try this then:
index=loadbalancer r_host="sport.mtm.com" req="/api/v2/log/exception" earliest=-2h latest=-1h | stats count | appendcols [ search index=loadbalancer r_host="sport.mtm.com" req="/api/v2/log/exception" earliest=-1h latest=now() | stats count as count2] | eval perc= round(((count2 - count) * 100 / count),2) | fields + perc | search perc < 50

0 Karma

amirarsalan
Explorer

Perfect should i still use "number of results is more than 0."

0 Karma

vishaltaneja070
Motivator

Yup correct.

please accept the answer so that thread can be closed

0 Karma

amirarsalan
Explorer

Thanks a lot

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...