Hi, is it possible to throttle alerts whilst conditions are sustained.
For example:
Search result per minute = 10, 9, 12, 8, 2, 4, 0, 0, 0, 0, 0, 0, 0, 1, 2, 4, 10, 10, 10, 8, 9, 3, 0 , 0 , 0, etc
So, I want my alert to be triggered at the zeros in bold, not for each subsequent one. So a service may stop running, but I don't want to receive constant reminders that it is not running, equally when I restart it again I want the alerts to become active again.
Yes you should be able to achieve that without even using throttling.
One way to do this is by using this search
.... | timechart span=1m count | eventstats sum(count) as total | tail 1 | search count=0 total!=0
This search will return one result if the most recent minute has no events but the previous one has some.
So here's the way to setup this alert based on "I have an int field called Fixtures in each event, and I want to know when it reaches zero, but not any subsequent zeros. So if(LastThreeMinutesHasFixtures && !LastTwoMinutesHasFixtures) ALERT"
compare last two minutes with first
.... | stats last(Fixture) AS FixtureThreeMinutesAgo, sum(Fixture) AS Total | where Total>0 AND Total==FixtureThreeMinutesAgo
FixtureThreeMinutesAgo - is the value of Fixture 3 minutes ago, remember that Splunk processes events in descending time order
Total - the sum of all three minutes, so if Total==FixtureThreeMinutesAgo then the last two minutes have no Fixture
Yes you should be able to achieve that without even using throttling.
One way to do this is by using this search
.... | timechart span=1m count | eventstats sum(count) as total | tail 1 | search count=0 total!=0
This search will return one result if the most recent minute has no events but the previous one has some.
I'm kind of walking away from this for now, but in the future I'd like to get it figured. I have an int field called Fixtures in each event, and I want to know when it reaches zero, but not any subsequent zeros. So if(LastThreeMinutesHasFixtures && !LastTwoMinutesHasFixtures) ALERT
The reason it is not working is because stats will return only one result - you need two, current minute and previous minute. Can you please restate your problem but now using Fixtures and Markets?
I have to add two fields together first. Not sure what 'doing the right thing' will look like. My search now ... | stats sum(Fixtures) as f sum(Markets) as m | eval count=f+m | timechart span=1m count | eventstats sum(count) as total | tail 1 | search count=0 total!=0
You should set the alert mode to once per search and send you an alert when the number of results is greater than 0. Can you verify that the search including the tail cmd is doing the right thing?
Seems to be sending me constant alerts. How should the options be setup? Conditions/Alert Mode etc???