Looking for solution of one alert calling script to execute second saved search

[burwell]

We want to do a search every minute on some logs. We want to identify those hosts whose events have http_code=5xx more than one percent of the time. And we want to see the actual events.

What I envisioned was one alert that would count the total events, count the 5xx events and when 5xx/total > 1% would alert and call a script that would perform a second search on the very same time event and show us the hosts and events that are involved.

We tried various things including evenstats, but it is too slow trying to do this in one search every minute. The search took too long.

I'm looking for a script called by search1 where I can call a named saved search2 with earliest/latest that I get from search1 and then have it alert.

Or maybe this is too complicated and there is a simpler way to go about this. Thanks!

[burwell]

If you answer the question I can then accept the answer...

[dolivasoh · Accepted solution]

Do you need to see all the events containing a 5xx error or would a sample do? if so, experiment with something like

| stats latest(_raw) as sample_event

If that's no good, I might suggest using a subsearch with append or appendcols

[burwell]

subsearch is the solution for me. I had never really used it before. That is exactly what I wanted. Thanks!