Alerting

How to create an alert in Splunk for Event A only if Event B doesn't also exist in a 30 second interval?

Magnus_001
Explorer

Hello,

I have an application that logs a "appsvr disconnected" and a "appsvr connected" message in the app log.  I have created the extraction called connectionstatus to indicate as such.  I would like to create a Splunk alert to notify me only when the connectionstatus = "appsvr disconnected" appears in the log and it is not followed by a connectionstatus = "appsvr connected" in a 30 second window.  I am trying to cut down on the false positives, it seems fairly straight forward but I haven't been able come up with a search that would satisfy this condition.  Any help is greatly appreciated.  Thanks!

Conditions:

connectionstatus="appsvr disconnected" appears in the log by itself for 30 seconds or more - Alert.

 connectionstatus="appsvr disconnected" appears in the log but the connectionstatus="appsvr connected" also appears in a 30 second window - False positive, don't alert.

-Magnus

Labels (2)
0 Karma

to4kawa
Ultra Champion

sample:

|makeresults count=2
| streamstats count
| eval _time=if(count=2,relative_time(_time,"-1d@h"),relative_time(_time,"@h"))
| makecontinuous span=5s _time
| eval connectionstatus="appsvr ".mvindex(split("disconnected,connected",","),random() % 2)
| streamstats count
| where count % 2 = 1
| autoregress connectionstatus as p
| where p!=connectionstatus
| rename COMMENT as "this is sample. from here, the logic"
| streamstats count(eval(connectionstatus="appsvr disconnected")) as session
| streamstats range(_time) as duration by session
| stats list(eval(strftime(_time,"%FT%T"))) as times max(duration) as duration list(connectionstatus) as connectionstatus by session
| sort session
| where duration > 30
0 Karma

isoutamo
SplunkTrust
SplunkTrust

I would like to add "where (duration > 0 OR duration == 0) to @to4kawa answer to catch cases where there is only disconnected but not connected after that.  

r. Ismo

0 Karma
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...