Alerting

How to create an alert scheduled in realtime and only triggered once per result?

christianubeda
Path Finder

Hi team!

I have problems with my alerts in realtime. I have like 70 and most of them don't trigger when they should.

I try to do schedules in realtime (Run the alert every 5 min) to avoid realtime problems

But it still misses events...

If I set run crontab every 5 min in 1h window it sends me tons of mail with repeat alerts.

What can I do?

I need an alert in real time that doesn't miss events.

alt text

0 Karma

snigdhasaxena
Communicator

Hi @christianubeda

When you run alert in real time, alerts are skipped/ do not get triggered because of the alerts in queue.
Suppose there's a huge traffic while splunk generates alert for current event, there may be several other alerts in queue since it is running for real time. this in turn leads splunk to not to generates alerts due to too many searches in pipeline (skipped searches).

My suggestions :
1. Run alert as scheduled search using cron schedule.
2. Pick a wider time range
3. You can use throttle to avoid duplicate alerts

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...