topic Re: How to use rex extracted field value in another search in Splunk Search

How to use rex extracted field value in another search

gcharles — Fri, 28 Jun 2019 07:19:58 GMT

I have a search like the following:
index="trans" source="logfilename" "ErrorCode=81009" requestid = "*ABC*" | rex field=requestid "-(?.*)$"

This search finds me instances of the 81009 error and then extracts a UID from the request ID (request IDs look like abc3232-ABC123ab45.

I would then like to take the values extracted, now stored in the UID field and perform a whole different search:

index="secondarydata" app="payments-service" | rex field=breadcrumb "-(?.*)$"
(then something like the below)
| where UID = UID2

I am trying to take the UID from the first search and find data from a second search that also has that UID present.

I am not aquainted enough with join/subsearch and such but I assume I need to use something like that...

Re: How to use rex extracted field value in another search

gcusello — Fri, 28 Jun 2019 07:40:46 GMT

Hi gcharles,
You can extract the field using the Field Extractor or, having the regex, create a new field in the Fileds section.
Then you have two choices:

if you have to use it only in the same App, shate the field att App level,
if you have to use it also in other Apps, shate the field att Global level.

You have to connect the field to a sourcetyoe, check if the regex extract always the correct values.

Bye.
Giuseppe

Re: How to use rex extracted field value in another search

FrankVl — Fri, 28 Jun 2019 07:55:36 GMT

First implement these field extractions to be automatically done, instead of relying on rex.

Then you can do:

index="secondarydata" app="payments-service" [search index="trans" source="logfilename" "ErrorCode=81009" requestid = "*ABC*" | table UID]

The subsearch will run first and will result in (UID="abc" OR UID="def" OR ...) that is embedded in the outer search.

Re: How to use rex extracted field value in another search

gcharles — Mon, 01 Jul 2019 00:20:21 GMT

Sorry, can you explain more because it has not helped.

I can see that the subsearch will result in a table like the following:

UID
---------
ABC45678
23536352
asfsdfdef434
etc...

But how am I to then try and find events in the secondarydata index using anyone of the results in the table?

For instance (not worrying about field extract at the moment):

index=secondarydata app="payments-service" 
    [search index="trans" sourcetype=kvp" "ErrorCode=81009" requestid=*AU* 
    |  table requestid] camel.breadcrumbId = requestid

I just want to see the log events from the secondarydata index in the search results that have the same requestid's that were found in the subsearch.

Further, I would also like to do a wildcard match using the UID/requestid where the end of the search might be:
camel.breadcrumbId = *requestid*

Re: How to use rex extracted field value in another search

FrankVl — Mon, 01 Jul 2019 07:36:37 GMT

"not worrying about field extract"

As I mentioned: you need to have the UID field extracted properly (automatically, not with rex) before this solution can work.