Alerting

Splunk alert trigger issue

Kareem_Naeem
Loves-to-Learn

my alert is not triggered even with many matching events here are the details:

 


Screenshot from 2024-10-14 21-21-27.pngScreenshot from 2024-10-14 21-28-28.png

while the activity that generate these logs is running the real time alert is processing and found those events in the screenshot. i have waited for 5 mins and same issue

i have also tries scheduled and still the same issue no triggering

Labels (1)
0 Karma

Kareem_Naeem
Loves-to-Learn

and here i have tried the scheduled alert and here is the logs related to that alert:

log 1:

10-14-2024 22:16:01.088 +0400 INFO SavedSplunker - AlertNotifier::notifySearchCompleted: called for sid=scheduler__kareem__search__kareem_at_1728929760_35, condition=1


log 2:


10-14-2024 22:16:01.809 +0400 INFO SavedSplunker - savedsearch_id="nobody;search;kareem", search_type="scheduled", search_streaming=0, user="kareem", app="search", savedsearch_name="kareem", priority=default, status=success, digest_mode=1, durable_cursor=0, scheduled_time=1728929760, window_time=0, dispatch_time=1728929760, run_time=0.178, result_count=1531, alert_actions="email", sid="scheduler__kareem__search__kareem_at_1728929760_35", suppressed=0, action_time_ms=718, thread_id="AlertNotifierWorker-0", workload_pool=""

0 Karma

PickleRick
SplunkTrust
SplunkTrust

1. Please, don't use real-time alerts. Except for very very rare cases real-time searches should be avoided altogether.

2. How did you confirm that the alerts weren't triggered? Is the email the only action you have defined? Maybe the problem is with your action, not the alert itself.

0 Karma

Kareem_Naeem
Loves-to-Learn

email was the only action 

and how did i know is by checking the triggered alerts secion under activity, and as i said i tried scueduled alerts the same issue 

 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

You must explicitly add an action to add an alert to triggered alerts. So if your only action was email, you must check why the email wasn't delivered (look in _internal for sendemail.py). At first glance your logs suggest that the alert notifier was actually dispatched.

0 Karma

Kareem_Naeem
Loves-to-Learn

i have had an error in the email logs and i have fized it and now im receiving emails correctly in my local mail server but still the alert when triggered it's not shown in the triggered alerts 

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Automating Threat Operations and Threat Hunting with Recorded Future

    Automating Threat Operations and Threat Hunting with Recorded Future June 29, 2026 | Register   Is your ...

Keep the Learning Going with the New Best of .conf Hub

Hello Splunkers, With .conf26 getting closer, there’s already a lot of excitement building around this year’s ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...