Splunk Search

How to trigger an alert when status field is true for more than 5 min no matter the amount of event

Cheng2Ready
Communicator

I have search query, if the Status is field is true for more than 5 min, I need to trigger an alert  no matter the Event count result. if its within the timeframe then fire.
Mabey even have it search for every 1minute.


for example  this should not fire an Alert because it recovered within the 5 min

1:00 Status = Down   (event result count X5)
1:03 Status = up
1:07 Status = Down  (event count X3)
1:10 Status = up
1:13 Status = up
1:16 Status = up

for example  this should  fire an Alert 

1:00 Status = Down  (event result count X1)
1:03 Status = Down (event result count X1)
1:07 Status = Down (event result count X1)
1:10 Status = up
1:13 Status = up
1:16 Status = up

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Your data does not match your description - the Status field appears to be either "up" or "Down" not "true" - because of this, it is not clear whether you want an alert if there has been a period of at least 5 minutes of Status being "Down" or Status being "up" anywhere within the time period of the search - please clarify your requirement

0 Karma

Cheng2Ready
Communicator

@ITWhisperer This is what I imagine it should look like 
but im not sure if there is a way to add in a condition for Streamstats 
for this command?  or a workaround?
"reset_on_change= if (status="UP", 1, 0)  "

| bucket span=1m _time
| eval status_change=if(status="DOWN",1,0)
| streamstats sum(status_change) as down_count  reset_on_change= if (status="UP", 1, 0)
| eval is_alert=if(down_count >=5 AND status="DOWN",1,0)
| where is_alert=1

0 Karma

Cheng2Ready
Communicator

@ITWhisperer want an alert if there has been a period for every1 minute of at least 5 minutes of Status being "Down" and if its interrupted with a status = Up then it resets the count and will not alert regarding the amount of event counts

0 Karma
Get Updates on the Splunk Community!

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...

Cloud Monitoring Console - Unlocking Greater Visibility in SVC Usage Reporting

For Splunk Cloud customers, understanding and optimizing Splunk Virtual Compute (SVC) usage and resource ...

Automatic Discovery Part 3: Practical Use Cases

If you’ve enabled Automatic Discovery in your install of the Splunk Distribution of the OpenTelemetry ...