Splunk Search

How can you emulate a sub-subsearch?

Lowell
Super Champion

Is there anyway of emulating a nested subsearch? I know its sometimes possible to rewrite a search to factor-out a subsearch, but I'm hitting a wall.

Yes, I know this could be possible using lookups, but for multiple reasons I don't want to go there. This is a rather infrequent search and I would have to rebuild the lookup frequently (because out of date data is worst than no data, in this case). So right now, it's just not worth the overhead and maintenance.

Here is search I really want to use: (BTW, this search does run, but it doesn't return any results. I suspect that this is is because arbitrarily deep nested sub-searches are not supported.

eventtype="wm-tn-doc-status-change"  [ search eventtype="wm-tn-doc-related" "EDIINT PAYLOAD" [ search tag::host=production sourcetype=wm_is_server B2B Invoice transmissionId "transmission sent" bizdoc | fields bizdoc | rename bizdoc as search | format ] ] | rex "System status is (?<system_status>.*?); user status is (?<user_status>.*?)\.[\r\n]" | stats last(user_status) as user_status, last(eval(strftime(_time, "%Y-%m-%d %H:%M"))) as timestamp

Working from the inside out, here is the information I'm trying to get:

The first search finds the primary document id:

tag::host=production sourcetype=wm_is_server B2B Invoice transmissionId "transmission sent" bizdoc

The second search takes that first document id, and finds a secondary document that related to the first in a specific way:

eventtype="wm-tn-doc-related" "EDIINT PAYLOAD"  <Insert level-1 doc id(s)>

Then the final (outer search) uses the secondary doc id to find a the most recent "status" update associated with that 2nd doc id:

eventtype="wm-tn-doc-status-change" <Insert level-2 doc id(s)> | stats first(status) ....

Any suggestions on how to unroll this search?

Tags (2)
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

I have definitely used two-level deep subsearches, so that shouldn't be a problem. I don't know about arbitrary levels of subsearch.

View solution in original post

Lowell
Super Champion

I got it working! Turns out my issue was that I was simply missing | fields ... | format in my outer sub-search, whoops! What a rookie mistake!

On a positive note, I did find out that you can do nested subsearches? Who knew? (A big thanks to Gerald for pointing this out!)

For anyone who cares, here is my final search:

eventtype="wm-tn-doc-status-change" [ search tag::host=production eventtype="wm-tn-doc-related" "EDIINT PAYLOAD" [ search tag::host=production sourcetype=wm_is_server B2B Invoice transmissionId "transmission sent" bizdoc | fields bizdoc | rename bizdoc as search | format ] | fields RelatedDocId | dedup RelatedDocId | format ]  | rex "System status is (?<system_status>.*?); user status is (?<user_status>.*?)\.[\r\n]" | stats last(user_status) as user_status, last(eval(strftime(_time, "%Y-%m-%d %H:%M"))) as timestamp by RelatedDocId

Unfortunately, after all this work, I've determined that I can't actually use this search the way I originally intended to. As it turns out, I really need access to that first "bizdoc" value which I lose immediately after the first subsearch; so I'll have to play around with some other ideas, or re-query the same events, or something...

gkanapathy
Splunk Employee
Splunk Employee

I have definitely used two-level deep subsearches, so that shouldn't be a problem. I don't know about arbitrary levels of subsearch.

View solution in original post

Lowell
Super Champion

I did some more testing and found the problem in my search. That's for point out that nested subsearches do work!

0 Karma

Stephen_Sorkin
Splunk Employee
Splunk Employee

For something this complex, I'd suggest making a custom python search command. Since the python command is given an authentication token, it should be able to run the first search, render it as desired and run the second and finally render and run the third search.

Lowell
Super Champion

Good point. I'm not quite ready do go to that level of effort quite yet. But I'll keep that option in mind. Thanks.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.