Nested query

[vihshah]

Hi,

So my task is to extract a field from a query and search for that field. That query will give an object value as a string and want to extract data from there. 

In summary, I need 3 things
1.  plain query to get the data and extract a particular field.
2. Use that field as an input for the second query.
3. Get object data as a string as a result, extract fields from there, and generate a report from it in tabular format.

I was able to reach till 1st step and extract the field from it.  but I am unable to search for it.
below is the query I tried.

sourcetype="mykube.source" "failed request"  | rex "failed request:(?<request_id>[\w-]+)" | table request_id | head 1 | eval req_query = request_id | search req_query

if I try till `head 1` I get first request_id but after that result is empty for me.

[vihshah]

okay, let me try

[gcusello]

Hi @vihshah ,

let us know if we can help you more, or, please, accept one answer for the other people of Community.

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉

[vihshah]

Hi @gcusello ,

regarding #2, I added the fields I was looking for, I tried the below query

sourcetype="mykube.source" "failed request"  
| rex "failed request:(?<request_id>[\w-]+)" 
| table _time request_id user_name

 however, I did not get `user_name` as my output. the reason was user_name was not populated as part of my first search. user_name gets populated when I do search by the `request_id` (but this request_id is not known to me before first search).

Issue here is, I can not write 2 separate queries, I need to club both the queries to get the final result

[gcusello]

Hi @vihshah,

let me understand: user_name is present only when there's also request_id?

If this is your requirement, please try this:

sourcetype="mykube.source" "failed request"  
| rex "failed request:(?<request_id>[\w-]+)" 
| search request_id=*
| table _time request_id user_name

Ciao.

Giuseppe

[vihshah]

Hi @gcusello ,

little correction, user_name is displayed in logs when we search with request_id. I used what you have given, but it is not displaying user_name still, so I removed the last 

| table _time request_id user_name

to see if it is displaying  user_name, but it is not displaying user_name, so I think below one is not searching based on the request_id 

| search request_id=*

[gcusello]

Hi @vihshah ,

check what are the conditions for user_name with:

sourcetype="mykube.source" "failed request"  
| rex "failed request:(?<request_id>[\w-]+)" 
| search user_name=*
| table _time request_id user_name

or

sourcetype="mykube.source" "failed request"  
| rex "failed request:(?<request_id>[\w-]+)" 
| search user_name=* request_id =*
| table _time request_id user_name

Ciao.

Giuseppe

[vihshah]

 @gcusello okay,

I think I know my issue, my user_name is part object string like below

OrderDetails{userId:"1", user_name:"A"}

if this is the case, how can I search it?

[gcusello]

Hi @vihshah ,

I don't know how the fiels is named, maybe "OrderDetails.user_name", see in interesting field its correct name.

If instead the issue is the extraction of the field, it seems to be a json log, so you could use spath to extract fields.

If instead you want to use a regex, please share a sample of your logs.

Ciao.

Giuseppe

[vihshah]

Hi @gcusello , @PickleRick ,

Sorry for late response, my main issue is I want to use output of a query as input for the subsequent, more or like example given here
https://docs.splunk.com/Documentation/Splunk/9.2.0/Search/Aboutsubsearches
in how subsearches work example.

I want to extract failed request happened past 24 hours so I am trying to do something like below

sourcetype="mysource"  [search sourcetype="mysource" "failed request:(?<request_id>[\w-]+=" | table request_id | top limit=100 request_id]

but this supposed to give me 100 failed req (because I have it in logs). I am not able to extract by above query

[PickleRick]

1. The general idea is sound but

2. The "top" command returns rows containing a value, count of events with this value and a percentage in the whole sample. So your subsearch will get rendered as

((request_id="something" AND count=something AND percent="something")
OR (request_id="something" AND count="something" AND percent="something")
OR [...] )

So as you most probably don't have matching values in your data, you won't find anything. If you want to return only the request_id values from your subsearch, you must further limit the list of your returned fields from the subsearch by adding "fields" or "table" command at its end.

[ITWhisperer]

Please check your subsearch as it looks like it isn't extracting anything. It is best to paste your search string into a codeblock </> so that it doesn't get reformatted and lose potentially vital information

[vihshah]

Hi @ITWhisperer , @PickleRick ,

adding it again

sourcetype=“my_source” [search sourcetype="my_source" "failed request, request id=" | rex “failed request, request id==(?<request_id>[\w-]+)" | top limit=100 request_id | fields request_id]

so according subsearch document, my subsearch supposed to extract failed first 100 request and it should work as my main search and should search for that 100 request.  but this is not happening

[ITWhisperer]

It is not possible to tell whether your subsearch should work or not since, despite being asked before, you have not shared your events (anonymised of course). If you want further assistance, please share some sample events preferably in a code block </> to prevent loss of vital information.

[vihshah]

sorry, I did not get, what should I share?

[ITWhisperer]

Some of the events as returned by your subsearch

sourcetype="my_source" "failed request, request id="

[vihshah]

this is how sample event looks like

2024:02:09:13:47:07.078 ERROR boundedElastic-6362 c.v.v.h.UserErrorHandler -24jan-rre2655-5b684rfb9b-jcfd4  failed request, request id=0a1-0b2-0a3, error: ValidationException{message=\'Status: 915 User not part of registery.  at least one seller did not have the same user id as the initial OT1 inquiry user, OT1 user=[user id=985238, seller id=134550], all merchants=[(user id=10, seller id=20), (user id=10, seller id=20), (user id=10, seller id=20), (user id=10, seller id=20)]\’}

now my task is to extract that request id and extract the events of it

[ITWhisperer]

Your event data has only one = after request id, not two as you have used in your rex. Try this

sourcetype=“my_source” [search sourcetype="my_source" "failed request, request id=" | rex “failed request, request id=(?<request_id>[\w-]+)" | top limit=100 request_id | fields request_id]

[vihshah]

what is difference between my above query and the query your provided?, I did the same thing, right?

[vihshah]

I see, there was missing "+" .  but I got same empty results

[ITWhisperer]

Not sure where the missing + is coming from but this shows you had two = in your search

[ITWhisperer]

As I said, you have two = in your regex, your event only has one