Alerting

How to create an alert in Splunk for Event A only if Event B doesn't also exist in a 30 second interval?

Magnus_001
Explorer

Hello,

I have an application that logs a "appsvr disconnected" and a "appsvr connected" message in the app log.  I have created the extraction called connectionstatus to indicate as such.  I would like to create a Splunk alert to notify me only when the connectionstatus = "appsvr disconnected" appears in the log and it is not followed by a connectionstatus = "appsvr connected" in a 30 second window.  I am trying to cut down on the false positives, it seems fairly straight forward but I haven't been able come up with a search that would satisfy this condition.  Any help is greatly appreciated.  Thanks!

Conditions:

connectionstatus="appsvr disconnected" appears in the log by itself for 30 seconds or more - Alert.

 connectionstatus="appsvr disconnected" appears in the log but the connectionstatus="appsvr connected" also appears in a 30 second window - False positive, don't alert.

-Magnus

Labels (2)
0 Karma

to4kawa
Ultra Champion

sample:

|makeresults count=2
| streamstats count
| eval _time=if(count=2,relative_time(_time,"-1d@h"),relative_time(_time,"@h"))
| makecontinuous span=5s _time
| eval connectionstatus="appsvr ".mvindex(split("disconnected,connected",","),random() % 2)
| streamstats count
| where count % 2 = 1
| autoregress connectionstatus as p
| where p!=connectionstatus
| rename COMMENT as "this is sample. from here, the logic"
| streamstats count(eval(connectionstatus="appsvr disconnected")) as session
| streamstats range(_time) as duration by session
| stats list(eval(strftime(_time,"%FT%T"))) as times max(duration) as duration list(connectionstatus) as connectionstatus by session
| sort session
| where duration > 30
0 Karma

isoutamo
SplunkTrust
SplunkTrust

I would like to add "where (duration > 0 OR duration == 0) to @to4kawa answer to catch cases where there is only disconnected but not connected after that.  

r. Ismo

0 Karma
Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Dynamic formatting from XML events

This challenge was first posted on Slack #puzzles channelFor a previous puzzle, I needed a set of fixed-length ...

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  🚀 Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...