Hello,
I would like to be able to raise an alert when the status of a condition changes?
For example:
- when the CPU user percentage is higher than 80% I would like to raise an alert that launch a command raising the alarm_on to an external program. This should be done only if the previous CPU user percentage was lower than 80%. There is no need to raise the alarm again.
- when the CPU user percentage goes back to below 80% I would like to send a command to my external program that set the alarm_off. This has to be done only if the previous CPU user percentage was higher than 80%.
Do you know how can I do it in Splunk?
I could raise an alert based on a search such as
sourcetype="cpu"| where CPU="all" and pctUser>20
and add the conditions to
sourcetype="cpu"| where CPU="all" and pctUser>20 | stats count=1
so that I am sure that the condition has been met only once.
The problem is:
- I should specify the time, like last 2 minutes
- How can I make sure that this is met by host?
Thanks for your help.
Cheers,
Adriana
Well, at the end the only way I found is to create one alert that raise the alarm when the following condition is met:
sourcetype=cpu earliest=-2m | multikv | where CPU="all" | stats count(eval(pctUser > 80)) as a ,count(eval(pctUser < 80)) as b, latest(pctUser) as c, by host | search a =1 AND b = 1 AND c>80
that runs every minute and it looks at the past 2 minutes.
In order to unraise the alarm, I created another alert that is launched when the following condition is met:
sourcetype=cpu earliest=-2m | multikv | where CPU="all" | stats count(eval(pctUser > 80)) as a ,count(eval(pctUser < 80)) as b, latest(pctUser) as c, by host | search a =1 AND b = 1 AND c<80
it also runs every minute and looks at the past 2 minutes.