How can you emulate a sub-subsearch?

[Lowell]

Is there anyway of emulating a nested subsearch? I know its sometimes possible to rewrite a search to factor-out a subsearch, but I'm hitting a wall.

Yes, I know this could be possible using lookups, but for multiple reasons I don't want to go there. This is a rather infrequent search and I would have to rebuild the lookup frequently (because out of date data is worst than no data, in this case). So right now, it's just not worth the overhead and maintenance.

Here is search I really want to use: (BTW, this search does run, but it doesn't return any results. I suspect that this is is because arbitrarily deep nested sub-searches are not supported.

eventtype="wm-tn-doc-status-change"  [ search eventtype="wm-tn-doc-related" "EDIINT PAYLOAD" [ search tag::host=production sourcetype=wm_is_server B2B Invoice transmissionId "transmission sent" bizdoc | fields bizdoc | rename bizdoc as search | format ] ] | rex "System status is (?<system_status>.*?); user status is (?<user_status>.*?)\.[\r\n]" | stats last(user_status) as user_status, last(eval(strftime(_time, "%Y-%m-%d %H:%M"))) as timestamp

Working from the inside out, here is the information I'm trying to get:

The first search finds the primary document id:

tag::host=production sourcetype=wm_is_server B2B Invoice transmissionId "transmission sent" bizdoc

The second search takes that first document id, and finds a secondary document that related to the first in a specific way:

eventtype="wm-tn-doc-related" "EDIINT PAYLOAD"  <Insert level-1 doc id(s)>

Then the final (outer search) uses the secondary doc id to find a the most recent "status" update associated with that 2nd doc id:

eventtype="wm-tn-doc-status-change" <Insert level-2 doc id(s)> | stats first(status) ....

Any suggestions on how to unroll this search?

[Lowell]

I got it working! Turns out my issue was that I was simply missing | fields ... | format in my outer sub-search, whoops! What a rookie mistake!

On a positive note, I did find out that you can do nested subsearches? Who knew? (A big thanks to Gerald for pointing this out!)

For anyone who cares, here is my final search:

eventtype="wm-tn-doc-status-change" [ search tag::host=production eventtype="wm-tn-doc-related" "EDIINT PAYLOAD" [ search tag::host=production sourcetype=wm_is_server B2B Invoice transmissionId "transmission sent" bizdoc | fields bizdoc | rename bizdoc as search | format ] | fields RelatedDocId | dedup RelatedDocId | format ]  | rex "System status is (?<system_status>.*?); user status is (?<user_status>.*?)\.[\r\n]" | stats last(user_status) as user_status, last(eval(strftime(_time, "%Y-%m-%d %H:%M"))) as timestamp by RelatedDocId

Unfortunately, after all this work, I've determined that I can't actually use this search the way I originally intended to. As it turns out, I really need access to that first "bizdoc" value which I lose immediately after the first subsearch; so I'll have to play around with some other ideas, or re-query the same events, or something...

[gkanapathy · Accepted solution]

I have definitely used two-level deep subsearches, so that shouldn't be a problem. I don't know about arbitrary levels of subsearch.

[Lowell]

I did some more testing and found the problem in my search. That's for point out that nested subsearches do work!

[Stephen_Sorkin]

For something this complex, I'd suggest making a custom python search command. Since the python command is given an authentication token, it should be able to run the first search, render it as desired and run the second and finally render and run the third search.

[Lowell]

Good point. I'm not quite ready do go to that level of effort quite yet. But I'll keep that option in mind. Thanks.