topic Re: How rex field list values assign dynamically to source path as subquery ? in Splunk Search

How rex field list values assign dynamically to source path as subquery ?

alok — Mon, 16 Nov 2020 22:41:33 GMT

Hello,

Query one returns a result with one fields as list of values. I want to pass those list of value as the search source path and result returns for second query. Given below is the detail.

Please suggest how to achieve ?

Query1 :

index="os" (source="/var/log/steps/*/controller") sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-109-*-*") | transaction source startswith=("/code/ttt_env.sh" OR "/code/ttt_gen.sh" ) endswith="startRun() called" | rex field=_raw "(?<step_function>\bs-[a-zA-Z0-9_]+)"

It does return the output and value of

Query1 Output :

step_function values listed as in field like : s-BBBUL8NJBYE45, s-AAAUL8NJBYEI3

Now these value I want to generate the further query using step_function values like ( Hard coded by hand it worked)

append [search index="os" source=("/var/log/steps/s-BBBUL8NJBYE45/stdout" OR /var/log/steps/s-s-AAAUL8NJBYEI3/stdout") sourcetype="too_small" (host="ip-101-108-*-*"" OR host="ip-101-108-*-*"*")]

How to perform dynamically and achieve this functionality without hardcoding.

Tried like this but didn't work

index="os" (source="/var/log/steps/*/controller") sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-108-*-*") |
transaction source startswith=("/code/ttt_env.sh" OR "/code/ttt_gen.sh") endswith="startRun() called" |
rex field=_raw "(?<rec_prod_step_function>\bs-[a-zA-Z0-9_]+)" | search rec_prod_step_function="*"
| append [search index="os" source="/var/log/steps/$rec_prod_step_function$/stdout" sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-108-*-*")]

Note : "/var/log/steps/$rec_prod_step_function$/stdout"

Thanks in advance.

Re: How rex field list values assign dynamically to source path as subquery ?

ITWhisperer — Mon, 16 Nov 2020 23:54:56 GMT

Rather than append, try using map

index="os" (source="/var/log/steps/*/controller") sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-108-*-*") | transaction source startswith=("/code/ttt_env.sh" OR "/code/ttt_gen.sh") endswith="startRun() called" | rex field=_raw "(?<rec_prod_step_function>\bs-[a-zA-Z0-9_]+)" | where rec_prod_step_function="*" | map search="search index=\"os\" source=\"/var/log/steps/$rec_prod_step_function$/stdout\" sourcetype=\"too_small\" (host=\"ip-101-108-*-*\" OR host=\"ip-101-108-*-*\")" maxsearches=0

Re: How rex field list values assign dynamically to source path as subquery ?

alok — Tue, 17 Nov 2020 03:09:33 GMT

I ran the suggested query getting a error message

Error in 'map': Did not find value for required attribute 'rec_prod_step_function'.

Please suggest.

As debug I break the query when I ran

It is not returning any event.

but when I used "where" to "search"

Query returns two events that is correct.

Please suggest.

Thanks !!

Re: How rex field list values assign dynamically to source path as subquery ?

ITWhisperer — Tue, 17 Nov 2020 07:50:13 GMT

If you get the right results with search instead of where, does the map function do what you want?

I don't understand why search works but where doesn't. Does the rec_prod_step_function field get extracted successfully? Can you provide the results of the successful query?