Alerting

Setting an alert for consecutive "errors" sorted by another variable

Paxxxman
Explorer

Hi,

I'm trying to set up an alert that checks for three '0' values in a row from the same "lane". If a non-zero value is found in between no alert is made. The transaction coming through from the logs looks like:

NAME=2012_07_20_07_59_56_729_0_00004_00002.jpg,OCR=BJH27V,CONFIDENCE=563,STATE=NSW,LANE=4

Basically, whenever there's three CONFIDENCE=0 in a row in the same LANE, I need an alert to be sent. I tried following some steps from another question posted here which used streamstats, but I'm not sure how to use it and also differentiate by lane.

All help is appreciated! Thanks!

Tags (1)
0 Karma

gkanapathy
Splunk Employee
Splunk Employee

With realtime search, you could use:

... | dedup 3 LANE | stats count(CONFIDENCE==0) as count | where count==3

In a non-realtime periodic search, use:

... | streamstats current=t window=3 global=f
        count(CONFIDENCE==0) as count
      by LANE
    | where count==3

Then you can either just use a per-result alert for each LANE value (which you really should do for the realtime version), or get an alert whenever you get more than 0 results from this search.

Paxxxman
Explorer

I tried the non-realtime periodic search and I've yet find any results from it. I know there exists plenty of times where there's 3 CONFIDENCE=0 in a row. Not sure what else I can try.

0 Karma
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...