Alerting

Alert on |search field=0 returns too many rows?

bmgilmore
Path Finder

I've got a scheduled search that calculates the variability of a numeric field over time that should always be moving. If I pipe a "search Variability=0" at the end, and run in search view, the search runs, shows a lot of rows at first as it calculates back through time, and then shows the correct number of rows.

Oddly, when I set this search to alarm, it sends many alarms and the CSV search results attached show many (almost all) of the available rows, each with a Variability of 0. Returning to the triggered job returns 0 (or whatever 1 or 2) rows as I would expect. Especially strange is that each time the alert triggers, it is a new number of false positive rows.

It's almost as if the search is not waiting to complete before triggering the alert? Any ideas here? Thanks in advance!

Tags (1)
0 Karma

jonuwz
Influencer

You probably need a custom condition in your alert

i.e.

main search

... | stats count(eval(Variability==0)) as not_variable

then a custom condition where

search not_variable > 0
0 Karma

MHibbin
Influencer

attached? ... you will need to paste it in the question as code (using the button "101010", or by starting each line of the code with 4 spaces, or by inclosing in backticks).

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...