Create an alert (Splunk query) for different nodes where if the status of the node goes down and doesn't come up within 1 hour then an alert should trigger.

[Inayath_khan]

Hi Guys,

I am Just creating a rule for a switch for multiple nodes where if the status of the switch goes down and doesn't comes up within an hour then it has to be triggered. But also if you see logs the status is getting up within a fraction of sec so i just want to put a threshold of 1 hour. Kindly help me on forming the Splunk query.

2019-12-02T17:25:38.448Z x.x.x.x <45>12376292: 12377249: *Dec 2 18:14:15.138: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up

2019-12-02T17:25:38.448Z x.x.x.x <45>12376291: 12377248: *Dec 2 18:14:15.101: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to down

Thanks in advance

[to4kawa · Accepted solution]

index=* Interface changed state to down 
|rex ".*Interface\s(?<interface>[\S]+),"
|rex ".*state\s+to\s+(?<vendor_action[\S]+)" 
| table _time,src_interface,vendor_action
| reverse
| streamstats count(eval(status="down")) as session by interface
| streamstats count(eval(status="start")) as start by session interface
| where start < 1 

[Inayath_khan]

Thanks @to4kawa it dint work. The status of an interface is getting up within 30 sec. I want a query that will trigger if status of the port is down for an interface for more than an hour and it's not up. Can we do some kind time comparision??

[to4kawa]

within 1 hour then an alert should trigger. is your question.
if status of the port is down for an interface for more than an hour and it's not up. is your comment and the reason not to accept
I create for your question. not to your problem. sorry.

[to4kawa]

what's your field extractions?

[Inayath_khan]

Hi @to4kawa this is my extraction,

index=* Interface changed state to down |rex ".*Interface\s(?[\S]+),"|rex ".*state\s+to\s+(?[\S]+)" | table _time,src_interface,vendor_action