Splunk Search

How rex field list values assign dynamically to source path as subquery ?

alok
Loves-to-Learn Everything

Hello,

Query one returns a result with one fields as list of values. I want to  pass those list of value as the search source path and result returns for second query. Given below is the detail.

Please suggest how to achieve ? 

Query1 : 

index="os" (source="/var/log/steps/*/controller")  sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-109-*-*") | transaction source startswith=("/code/ttt_env.sh" OR "/code/ttt_gen.sh" ) endswith="startRun() called" | rex field=_raw "(?<step_function>\bs-[a-zA-Z0-9_]+)"

It does return the output and value of 

Query1 Output : 

step_function values listed as  in field like : s-BBBUL8NJBYE45s-AAAUL8NJBYEI3

Now these value I want to generate the further query using step_function values like ( Hard coded by hand it worked)

append [search index="os" source=("/var/log/steps/s-BBBUL8NJBYE45/stdout" OR /var/log/steps/s-s-AAAUL8NJBYEI3/stdout")  sourcetype="too_small" (host="ip-101-108-*-*"" OR host="ip-101-108-*-*"*")]

How to perform dynamically and achieve this functionality without hardcoding. 

Tried like this but didn't work 

index="os" (source="/var/log/steps/*/controller") sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-108-*-*") |
transaction source startswith=("/code/ttt_env.sh" OR "/code/ttt_gen.sh") endswith="startRun() called" |
rex field=_raw "(?<rec_prod_step_function>\bs-[a-zA-Z0-9_]+)" | search rec_prod_step_function="*"
| append [search index="os" source="/var/log/steps/$rec_prod_step_function$/stdout" sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-108-*-*")]

Note : "/var/log/steps/$rec_prod_step_function$/stdout"

Thanks in advance.

Labels (3)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Rather than append, try using map

index="os" (source="/var/log/steps/*/controller") sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-108-*-*") |
transaction source startswith=("/code/ttt_env.sh" OR "/code/ttt_gen.sh") endswith="startRun() called" |
rex field=_raw "(?<rec_prod_step_function>\bs-[a-zA-Z0-9_]+)" | where rec_prod_step_function="*"
| map search="search index=\"os\" source=\"/var/log/steps/$rec_prod_step_function$/stdout\" sourcetype=\"too_small\" (host=\"ip-101-108-*-*\" OR host=\"ip-101-108-*-*\")" maxsearches=0
0 Karma

alok
Loves-to-Learn Everything

I ran the suggested query getting a error message 

Error in 'map': Did not find value for required attribute 'rec_prod_step_function'.

Please suggest.

As debug I break the query when I ran 

 

index="os" (source="/var/log/steps/*/controller") sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-108-*-*") |
transaction source startswith=("/code/ttt_env.sh" OR "/code/ttt_gen.sh") endswith="startRun() called" |
rex field=_raw "(?<rec_prod_step_function>\bs-[a-zA-Z0-9_]+)" | where rec_prod_step_function="*"

 

It is not returning any event.

but when I used "where" to "search"

 

index="os" (source="/var/log/steps/*/controller") sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-108-*-*") |
transaction source startswith=("/code/ttt_env.sh" OR "/code/ttt_gen.sh") endswith="startRun() called" |
rex field=_raw "(?<rec_prod_step_function>\bs-[a-zA-Z0-9_]+)" | search rec_prod_step_function="*"

 

Query returns two events that is correct.

Please suggest.

Thanks !!

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

If you get the right results with search instead of where, does the map function do what you want?

I don't understand why search works but where doesn't. Does the rec_prod_step_function field get extracted successfully? Can you provide the results of the successful query?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...