Alerting

How to send alert with 2 condition and time constraint.

tanzhiyu
Loves-to-Learn Lots

Query 1

index=ops_gtosplus trans_id="PREGATE_DOCU" application_m="GTOSPLUS_OPS_GATEGW_BW" msg_x="MSG PROCESSING | END OK"


Query 2

index=ops_gtosplus trans_id="PREGATE_DOCU" application_m="GTOSPLUS_OPS_GOS_SB" msg_x="MSG PROCESSING | END OK"



But query contain event_id. Want to know how to search records for event_id that is in query 1 but not in query 2. And need to give in 15sec allowance.

For e.g. event id appear in query 1 at 2pm. then if by 2:00:15pm, the event id still does not appear in query 2, need to send out alert.

Labels (1)
0 Karma

haventgotwifi
Engager

I would use the "latest=-15s" command in your search. This should achieve what you want

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @tanzhiyu,

let me understand: do you want to ches the event_ids that are in search1 but not in search2, is it correct?

if this is your need, try this:

index=ops_gtosplus trans_id="PREGATE_DOCU" (application_m="GTOSPLUS_OPS_GATEGW_BW" OR application_m="GTOSPLUS_OPS_GOS_SB") msg_x="MSG PROCESSING | END OK"
| stats dc(application_m) AS dc_application_m values(application_m) AS application_m BY event_id
| where dc_application_m=1 AND application_m="GTOSPLUS_OPS_GATEGW_BW"
| table _time event_id

Ciao.

Giuseppe

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @tanzhiyu,

sorry I forgot this condition, so, let me understand: do you want to check that in the following 15 seconds after the first event there isn't the second, is this correct?

Using my search you check that only the first message is present, you could run the search every 15 seconds so you're sure, using my search that the second condition isn't matched.

If instead you want to run your search for a larger period you have to use a different approach, using the transaction command:

 

index=ops_gtosplus trans_id="PREGATE_DOCU" (application_m="GTOSPLUS_OPS_GATEGW_BW" OR application_m="GTOSPLUS_OPS_GOS_SB") msg_x="MSG PROCESSING | END OK"
| transaction event_id startswith="GTOSPLUS_OPS_GATEGW_BW" endwith="GTOSPLUS_OPS_GOS_SB" keepevicted=true
| where duration>15
| table _time event_id

 

but this is a not performant search.

Ciao.

Giuseppe

 

0 Karma

tanzhiyu
Loves-to-Learn Lots

Hi,

Thanks for your solution. But I did not see any part of the query that is handling of delay of 15secs.

Because we are expecting some delay of 15secs, and dont want to receive false positive alerts to our duty personnel.

Thanks.

Zhi Yu

0 Karma
Get Updates on the Splunk Community!

Observability Unlocked: Kubernetes Monitoring with Splunk Observability Cloud

  Ready to master Kubernetes and cloud monitoring like the pros?Join Splunk’s Growth Engineering team for an ...

Wrapping Up Cybersecurity Awareness Month

October might be wrapping up, but for Splunk Education, cybersecurity awareness never goes out of season. ...

🌟 From Audit Chaos to Clarity: Welcoming Audit Trail v2

🗣 You Spoke, We Listened  Audit Trail v2 wasn’t written in isolation—it was shaped by your voices.  In ...