Alerting

Is it possible to have multiple alert conditions for one saved search?

shangshin
Builder

Hi,
One saved search can have only one alert condition.
I have "heartbeat" string in my log and I set up a saved search in scheduler --

 sourcetype="app.log" "Heartbeat check completed"

to trigger an alert email with one alert condition "if number of events is less than 1"

However, false alarms will occur when indexers are in the middle of restarting.

Is it possible to add an extra alert condition like scanning events are more than 10?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Assuming a scanning event looks like "scanning event", you could do this:

sourcetype="app.log" "Heartbeat check completed" OR "scanning event" | eval category = if(searchmatch("scanning event", "scanning", "heartbeat") | chart count over host by category | where scanning > 10 AND heartbeat < 5

Alert when there is at least one event, and it'll tell you hosts that had at least ten scanning events but less than five completed heartbeat checks.

martin_mueller
SplunkTrust
SplunkTrust

Something along the lines of host=* scanning=* NOT "heartbeat check completed"? Simple boolean expressions, really.

0 Karma

shangshin
Builder

Thanks a lot for the answer.

In most cases, I will get 1 row with 3 columns - host, heartbeat and scanning. However, assuming "heartbeat check completed" is not in the log due to application error, I will get only 1 row with 2 columns -- host and scanning. How can the alert be triggered in this case?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...