Alerting

How to create a search to find if a switch interface is down, then trigger an alert only if the interface is not running

Mohanveera1
Explorer

Hi Splunk Experts,

I tried to create the search but can't be successful in it, I need a search, if in case the interface of an Cisco switch is down and doesn't came to up state within 5 minutes it should throw an alert. The logs are as below screenshot. I tried to write the query only for one scenario i.e if the switch status changed state to down i can get the alert but i can't merge the both message fields back to back.

Required Alert scenario:

when first log containing the message field with changed state to down appears and within 5 minutes the next log containing the message field with changed state to up doesn't appear then i must get an alert.

 

Mohanveera1_0-1667470543576.png

 

Thanks in advance.....

 

Labels (4)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Alerts can't wait to see what will happen in the next few minutes.  They only look at what happened in the past.  This problem typically is solved by looking for the most recent event from each device.  If it's a "down" event and it's been at least 5 minutes then trigger an alert.

index=cisco_switch sourcetype=syslog:cswitch (message="changed state to down" OR message="changed state to up")
```Get the most recent event for each device```
| dedup device_IP
```Keep only down devices at least 5 minutes old```
| where (message="change state to down" AND _time<=relative_time(now(), "-5m"))
| table _time device_IP

Set the alert to trigger is the number of results is not zero.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...