Is there a way to chain searches?

[daniel333]

All,

A user just asked me this, any ideas on how to do this?

Splunkj Q: is the following supported?
I create an alert with a search trigger condition. If alert is triggered, run a different query, dump the results of that second query in an email to me
i.e., I want an alert to trigger a secondary search (because this additional search would contain useful diag. info on the main problem that triggered the alert originally) and have that secondary search data as the email (edited)

---

concretely
alert trigger: index=siebel "SBL-EAI-04117" earliest=-5m | stats count | where count>200 (edited)
if triggered, execute: index=siebel "SBL-EAI-04117" earliest=-60m | bin _time span=5m | stats count by _time | sort _time
(if error count threshold breached, show me the last 60 minutes of data in an email)

[pradeepkumarg]

You could do this in just one search

index=siebel "SBL-EAI-04117" earliest=-60m | bin _time span=5m | stats count by _time | sort _time | eventstats first(count) as last_5min | search last_5min > 200

[elliotproebstel]

Might be a better way, but this should work:

index=siebel "SBL-EAI-04117" earliest=-60m 
| bin _time span=5m 
| stats count by _time 
| sort _time
| eventstats latest(count) AS 5m_count
| where '5m_count'>200
| fields - '5m_count'

And then send an alert if the result count is not zero. The eventstats line will find the count from the latest 5 minutes, apply that count to all events, and then retain only the events where the 5m_count field is greater than 200. Because it'll be the same value for all events, it will either retain all events or throw away all of them.

[pradeepkumarg]

Saw your answer after submitting mine. Pretty close 🙂

[elliotproebstel]

Gives me more confidence that this is the right way! 🙂 Great minds and all that...