topic Re: consolidate the alert in Alerting

consolidate the alert

logloganathan — Tue, 29 Sep 2020 23:18:37 GMT

index=xyz host=a12fr* sourcetype = alert "A failed" OR "A success"
| head 1
| eval my_time=_time, current=Now()
| eval diff=current-my_time
| where diff>=100 AND like(_raw, "%failed%")

index=xyz host=a12fr* sourcetype = alert "B failed" OR "B success"
| head 1
| eval my_time=_time, current=Now()
| eval diff=current-my_time
| where diff>=100 AND like(_raw, "%failed%")

index=xyz host=a13fr* sourcetype = alert "B failed" OR "B success"
| head 1
| eval my_time=_time, current=Now()
| eval diff=current-my_time
| where diff>=100 AND like(_raw, "%failed%")

index=xyz host=a13fr* sourcetype = alert "A failed" OR "A success"
| head 1
| eval my_time=_time, current=Now()
| eval diff=current-my_time
| where diff>=100 AND like(_raw, "%failed%")

how to consolidate these alert to single alert?

Re: consolidate the alert

gowtham495 — Wed, 20 Feb 2019 13:03:33 GMT

Is success/failed is captured in any field name?

Re: consolidate the alert

logloganathan — Wed, 20 Feb 2019 13:15:11 GMT

no its not field name

Re: consolidate the alert

woodcock — Wed, 20 Feb 2019 13:49:08 GMT

Like this:

index=xyz (host=a12fr* OR host=a13fr*) AND sourcetype = alert AND (("A failed" OR "A success") OR ("B failed" OR "B success"))
| eval which = case(searchmatch("\"A failed\" OR \"A success\""), "A", searchmatch("\"B failed\" OR \"B success\""), "B", true(), "ERROR")
| dedup host which
| eval diff=_time - now()
| where diff>=100 AND like(_raw, "%failed%")

Re: consolidate the alert

richgalloway — Wed, 20 Feb 2019 13:54:07 GMT

Perhaps I'm missing something, but have you tried the obvious?

index=xyz (host=a12fr* OR host=a13fr*) sourcetype = alert (("A failed" OR "A success") OR ("B failed" OR "B success"))
| eval my_time=_time, current=Now() 
| eval diff=current-my_time 
| where diff>=100 AND like(_raw, "%failed%")

It can be refined further to this:

index=xyz (host=a12fr* OR host=a13fr*) sourcetype = alert ("A failed" OR "B failed") latest=-100s

Re: consolidate the alert

lakshman239 — Wed, 20 Feb 2019 13:59:29 GMT

You could do couple of things?
- assuming your raw event is similar, create a field to extract status - success /failed
- create eventtypes for each pattern . "A failed" OR "A success" , "B failed" OR "B success" etc..
- and possibly a macro to define hosts, say 'host A OR hostB OR host C etc.. to make search simpler
- then index=xyz (eventtype=A OR eventtype=B) | head .... your search...
- where possible avoid like(_raw, "%failed%") and bring that to your base search (before first pipe).

Re: consolidate the alert

logloganathan — Thu, 21 Feb 2019 10:47:43 GMT

if get both failure and success then i not want to display

Re: consolidate the alert

logloganathan — Thu, 21 Feb 2019 12:37:59 GMT

i am getting these error

Error in 'eval' command: Typechecking failed. 'OR' only takes boolean arguments.

Re: consolidate the alert

richgalloway — Thu, 21 Feb 2019 13:11:43 GMT

Then @woodcock has your answer.

Re: consolidate the alert

woodcock — Thu, 21 Feb 2019 13:47:00 GMT

I edited my answer and fixed that error.

Re: consolidate the alert

logloganathan — Fri, 22 Feb 2019 14:24:46 GMT

actually the requirement
if i get failure event then success event within 5 minutes..it should not be noted
if i get only failure event and no success event for last 5 minutes..it should be noted
if i get only success..it should not be noted
Could you please help me...

Re: consolidate the alert

logloganathan — Mon, 25 Feb 2019 10:30:56 GMT

@woodcock Could you please help

Re: consolidate the alert

lakshman239 — Mon, 25 Feb 2019 11:18:59 GMT

I think you are looking for success or failure, probably evenstats and streamstats could help. From line 3 in woodcock's response, try to change to eventstats and keep 5mins window and you can then do eventstats values(which) by host.

Pls look at https://www.splunk.com/blog/2014/04/01/search-command-stats-eventstats-and-streamstats-2.html and change as per your need.

Re: consolidate the alert

woodcock — Mon, 25 Feb 2019 16:34:15 GMT

I can help but let's move this to another new question and close out this one. Call me out in the new question and I will take a look at it.

Re: consolidate the alert

logloganathan — Mon, 25 Feb 2019 16:39:07 GMT

Sure woodcock