Alerting
Highlighted

Why isn't my splunk alert firing?

New Member

Hi,

I'm using Splunk Enterprise 7.1.1 and I'm having a problem with alerts.
I have tried to create an alert for the following search:

host="Alert_Trial"|noop|stats count|eval count = count + 1

I want the alert to send an email and I have configured my email settings as per the following link:
https://www.splunk.com/blog/2014/06/27/splunk-alerts-using-gmail-twitter-phone-calls-and-much-more.h...

I have triggered my alert to fire for Number of events greater than 0 and I have set it for every 5 minutes using Cron Schedule: */5 * * * *

I save the alert and it appears in my alerts list. But no longer how long I wait I never get an e-mail and the alert always says "There are no fired events for this alert". I have also tried using "Real-time" instead of Cron Schedule, but it still does not fire.

Can anyone help me resolve this?

Tags (3)
0 Karma
Highlighted

Re: Why isn't my splunk alert firing?

Splunk Employee
Splunk Employee

In order to troubleshoot this issue, kindly check below:

  1. Check triggered alerts from
    Settings > Knowledge > Searches, reports and alerts > Alerts column
    OR
    Activity > Triggered alerts

  2. Check under python.log for any error/warning message related to savedsearch/alert you want to trigger

  3. Also, you may need to check for the skipped searches. Maybe during skippedsearches time, you were running into your maxconcurrent limit, which is why this search was skipped multiple times and that is why you did not receive the alert.

Example of the log is as below:

INFO SavedSplunker - savedsearchid="nobody;SystemManage;SVaccount-authfail-emailsend", user="abcd", app="", savedsearchname="", priority=, status=skipped, reason="maxconcurrent limit reached", scheduledtime=1498555860, windowtime=0

In case if you see the above info message in logs, you should increase the limit for the maximum number of concurrent searches in limits.conf
For reference, kindly check: https://answers.splunk.com/answers/4888/getting-error-maximum-number-of-concurrent-searches-has-been...