Splunk Search
Highlighted

Send Alert if one event doesn't occur in 10 min

Explorer

I have used transaction and non transaction method.
Non transaction method yields result but the results are not being sent as an alert.
Could some one help how to achieve without using stats earliest latest commands, probably using if condition.
Example:

Event1: hostname slot 1 down
Event2: hostname slot 1 up

If event 2 doesn't arive in 10 min , I need to get an alert ,
Give examples if event 2 also be in different source type

Tags (4)
0 Karma
Highlighted

Re: Send Alert if one event doesn't occur in 10 min

SplunkTrust
SplunkTrust

Why is there the arbitrary constraint on not being able to use the stats earliest or latest commands? (It's not necessarily an issue in finding a solution, but I think many people who read this would like to know - have you had issues out of those before? Do you have a earliest/latest solution but someone's double-dared you to do it without?)

0 Karma
Highlighted

Re: Send Alert if one event doesn't occur in 10 min

Explorer

Dear Rich,
I have spent considerable amount of time to get an alert, till now no luck,
When run manually it shows the results but not as alert email(email never comes)

Transaction never works at all

Something splunk doesn't like,
I'll post exact search string

0 Karma
Highlighted

Re: Send Alert if one event doesn't occur in 10 min

Explorer

sourcetype=xyz ciscoLwappApIfDownNotify OR ciscoLwappApIfUpNotify OR bsnDot11StationAssociate | rex "CISCO-LWAPP-AP-MIB::cLApName.0

= STRING: (?<apname>\S+)" | rex "AIRESPACE-WIRELESS-MIB::bsnAPName.0 = STRING: \"(?<apname>\S+)?\"" | rex "CISCO-LWAPP-AP-MIB

::cLApDot11IfSlotId.0 = Gauge32: (?<radioslot>\S+)" | rex "AIRESPACE-WIRELESS-MIB::bsnStationAPIfSlotId.0 = INTEGER: (?<radioslot

>\S+)"| stats earliest(_time) as _time latest(_time) as time latest(snmpTrapOID_0) as action values(radioslot) as radioslot

values(cLApIfUpDownCause_0) as Cause values(cLApIfUpDownFailureCode_0) as failurecode values(cLApIfUpDownFailureType_0) as

failuretype values(cLApSysMacAddress_0) as apmac values(cLApAdminStatus_0) as adminstatus by apname | eval age=now()-time | where

age>300 AND action="OID: CISCO-LWAPP-AP-MIB::ciscoLwappApIfDownNotify"

I run this every 30 min for past 5 hrs, currently 2 results are shown but I don't receive as an alert

0 Karma
Highlighted

Re: Send Alert if one event doesn't occur in 10 min

SplunkTrust
SplunkTrust

“hostname slot 1 down” OR “hostname slot 1 up” indexearliest=-15m@m indexlatest=-5m@m

Trigger condition is if number of results are less than 2, corn schedule is

 0,10,20,30,40,50 * * * *

Assumes you only get one “down” event and don’t have any fields extracted.

0 Karma
Highlighted

Re: Send Alert if one event doesn't occur in 10 min

Explorer

Index earliest I tried I believe;
Still I have to use stats earliest and latest right?

0 Karma
Highlighted

Re: Send Alert if one event doesn't occur in 10 min

SplunkTrust
SplunkTrust

Not really. At least in most cases I can think of the only time you might really need stats latest or earliest would be to do “more work” on the results afterwards. The use case you presented seems like it has simpler solutions, like the one that jkat54 provided.

I would use a cron of */10 * * * *

0 Karma
Highlighted

Re: Send Alert if one event doesn't occur in 10 min

SplunkTrust
SplunkTrust

No, this search is going to run every 10 minutes and account for indexing latency of up to 5 minutes. If there are less than two events found, then it will trigger the alert.

0 Karma
Highlighted

Re: Send Alert if one event doesn't occur in 10 min

Explorer

Not working in my case

0 Karma
Highlighted

Re: Send Alert if one event doesn't occur in 10 min

SplunkTrust
SplunkTrust

If you run the search it's based on as a regular search, does that return results? Would any of those returned results indicate the alert should have fired?

0 Karma