Search based on results of another search?

[lnetherton]

Is it possible to perform a search on a whole dataset using a subset of terms from a previous search?

For example, I have a search that yields all the failed transactions based on an event type:

sourcetype=catalina* eventtype=search_fail

I get back rows in the following form:

Mar 6, 2014 12:24:38 AM api.core.helper.LogHelper severe SEVERE: There was an error for search_id=530959

These rows all contain a search_id number. I would like to then initiate a new search for any search_id equal to these values, so that I can see all events leading up to these failures. Is that possible?

I have tried various things like:

sourcetype=catalina* eventtype=search_fail | search search_id
sourcetype=catalina* eventtype=search_fail | search search_id=search_id
sourcetype=catalina* eventtype=search_fail [ search search_id ]
...

But nothing seems to do what I am after.

[lnetherton]

Actually, I've figured it out. I needed to use the fields operator to specifically select only the field that I want to use to search (search_id in my case). I also misunderstood the way that subsearches work -- it is important to know that the subsearch is evaluated first, and the result used to augment the outer search.

This search term ended up doing what I wanted:

sourcetype=catalina* [ search sourcetype=catalina* eventtype=search_fail | fields + search_id ]

It was useful to know that the sub-search operation implicitly appends a | format operator on to the end. Combined with the fields + search_id operation, the sub-search term is effectively expanded to something like this:

sourcetype=catalina* ( ( search_id="530959" ) OR ( search_id="529947" ) OR ( search_id="529938" ) OR ( search_id="529919" ) OR ( search_id="529793" ) OR ( search_id="529792" ) OR ( search_id="529568" ) OR ( search_id="529559" ) ) 

which of course produces the output that I was after.

[isoutamo]

Please remember that there are row and time out limits for sub searches.... Sometimes those will hit if you are have two many rows in sub search. In those cases you must divide this search to two part e.g. first create lookup and then queries values from it on the actual search.
R. Ismo

[slw07gdev]

You're the real MVP. 

[butzowj]

Wanted to thank you for this. I as able to use your example to solve a problem I had, and you also helped me understand how subsearches work!

Thanks!