Splunk Search

Why is my alert not triggered even though there are results that meet the criteria?

kotig
Path Finder

Hi All,

We have a search which checks for a total count of failures in system in the last 24 hours:

index=mydata earliest=-24h| stats list(Summary) as "Summary", count(Summary) as SummaryCount

when a specific failures occur, the SummaryCount has a value of 1 or more than one.

Based on that we have created an alert to trigger with custom trigger condition:
search SummaryCount>=1 and AlertType =Scheduled and is scheduled to run every day, one time at 1 am.
and the Action Options, set was "Once"

Every time we run our search we get a result, so i was expecting there will be 1 alert everyday until the SummaryCount value is returned. But i got the alert only on 1 day and later it does not send any alert emails.

Can you please help? This is really urgent.

Thanks
Koti

0 Karma
1 Solution

kotig
Path Finder

Thank you, appreciate your help on trying to help me. My alert is fixed now. I had a false alarm. Thanks everyone for your inputs.

View solution in original post

0 Karma

kotig
Path Finder

Thank you, appreciate your help on trying to help me. My alert is fixed now. I had a false alarm. Thanks everyone for your inputs.

0 Karma

lguinn2
Legend

I suggest that you should simply use the normal trigger condition of Number of Results > 0
Your custom search for the alert is broken in a couple of ways:

  • and must be capitalized as AND in searches if you want a boolean comparison.
  • Your search does not yield a field named AlertType

What this means is that your custom search will never return anything - so your alert will not fire.
As far as I can tell, you don't need a custom search for your alert at all. Change your settings as
suggested in the first paragraph and see what happens.

0 Karma

somesoni2
Revered Legend

How about you update your alert like this and try.

Search: index=mydata earliest=-24h| stats list(Summary) as "Summary", count(Summary) as SummaryCount | wher SummaryCount>0

Time Range: -1d@d to @d
Alert Type: Scheduled, Cron: 0 1 * * *
Alert condition: If number of events greater than 0
Action Options: Once

0 Karma

kotig
Path Finder

Sure thank you will try that option.

0 Karma

somesoni2
Revered Legend

Did you run the query for days you didn't get alert and found results matching your trigger condition?

0 Karma

kotig
Path Finder

Yes, i am getting value for the day it did not trigger.

0 Karma
Get Updates on the Splunk Community!

Splunk Security Content for Threat Detection & Response, Q1 Roundup

Join Principal Threat Researcher, Michael Haag, as he walks through:An introduction to the Splunk Threat ...

Splunk Life | Happy Pride Month!

Happy Pride Month, Splunk Community! 🌈 In the United States, as well as many countries around the ...

SplunkTrust | Where Are They Now - Michael Uschmann

The Background Five years ago, Splunk published several videos showcasing members of the SplunkTrust to share ...