topic Re: Recursive sub-search in Splunk Search

Recursive sub-search

rangarbus — Thu, 25 Jul 2024 19:35:42 GMT

I have 3 separate queries. I need to run them one after the other.

1. First query returns a field from each event that matches the search, say eventId

2. I need to make another query to identify events which has this eventId in the event , not a specific field. There will be zero or one row that will be returned in this case. I want to read a field on that event say "traceId".

3. Now i need to make a 3rd query using that returned traceId. There will be only one event. With the result returned, i need to fetch the "fileName" from that matched event.

This fileName is the final result that i need.

Any guidelines / example to do this.

Known issue: On the search 2, eventId from search 1 is not searchable as a field rather should be search on the _raw events as such. I tried sub-search , but always result on OR statement on a field. But i dont have such field on the _raw event for search 2.

Apologies if i sounded this confusing.

Re: Recursive sub-search

yuanliu — Thu, 25 Jul 2024 20:05:25 GMT

First, using subsearch should not be your first choice. Second, Splunk is not procedural; forced recursion on command will result in some unmaintainable code.

You need to provide additional information about your data in addition to that your second dataset doesn't have eventId readily extracted. I assume that the first "search" and second have different source types. I also assume that search period is roughly identical in all three. But I don't understand what is the dataset for the third "search". Is it yet another indexed source? Is it some sort of lookup table?

To ask answerable questions in this forum, follow the following golden rules that I call the Four Commandments:

Illustrate data input (in raw text, anonymize as needed), whether they are raw events or output from a search that volunteers here do not have to look at.
Illustrate the desired output from illustrated data.
Explain the logic between illustrated data and desired output without SPL.
If you also illustrate attempted SPL, illustrate actual output and compare with desired output, explain why they look different to you if that is not painfully obvious.

Re: Recursive sub-search

gcusello — Fri, 26 Jul 2024 06:40:55 GMT

Hi @rangarbus ,

you should try to run these three searches in nested mode starting from the third:

<third_search> [ search <second_search> [ search <first_search> | fields eventId ] | fields traceId ] | table fileName

if eventId must be searched as raw text because it isn't in a field called eventId, you could use this one:

<third_search> [ search <second_search> [ search <first_search> | rename eventId AS query | fields query ] | fields traceId ] | table fileName

I hope that this nested search will run on not so many events because it will not be so performant; if you'll have many events, you shuld accelerate each search in a summary index or in a Data Model.

Ciao.

Giuseppe

Re: Recursive sub-search

PickleRick — Fri, 26 Jul 2024 17:53:28 GMT

While technically it should be possible to do with @gcusello 's way of chaining subsearches it's a very bad idea. Subsearches do have their limitation so your result can be completely wrong.

Unfortunately if you really need to do a full text search it's not possible to use the techniques typically used in similar cases since they rely on common fields.

Be aware though that regardless of the subsearch use searching through unparsed data can also be very performance-intensive.