Splunk Search

How rex field list values assign dynamically to source path as subquery ?

alok
Loves-to-Learn Everything

Hello,

Query one returns a result with one fields as list of values. I want to  pass those list of value as the search source path and result returns for second query. Given below is the detail.

Please suggest how to achieve ? 

Query1 : 

index="os" (source="/var/log/steps/*/controller")  sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-109-*-*") | transaction source startswith=("/code/ttt_env.sh" OR "/code/ttt_gen.sh" ) endswith="startRun() called" | rex field=_raw "(?<step_function>\bs-[a-zA-Z0-9_]+)"

It does return the output and value of 

Query1 Output : 

step_function values listed as  in field like : s-BBBUL8NJBYE45s-AAAUL8NJBYEI3

Now these value I want to generate the further query using step_function values like ( Hard coded by hand it worked)

append [search index="os" source=("/var/log/steps/s-BBBUL8NJBYE45/stdout" OR /var/log/steps/s-s-AAAUL8NJBYEI3/stdout")  sourcetype="too_small" (host="ip-101-108-*-*"" OR host="ip-101-108-*-*"*")]

How to perform dynamically and achieve this functionality without hardcoding. 

Tried like this but didn't work 

index="os" (source="/var/log/steps/*/controller") sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-108-*-*") |
transaction source startswith=("/code/ttt_env.sh" OR "/code/ttt_gen.sh") endswith="startRun() called" |
rex field=_raw "(?<rec_prod_step_function>\bs-[a-zA-Z0-9_]+)" | search rec_prod_step_function="*"
| append [search index="os" source="/var/log/steps/$rec_prod_step_function$/stdout" sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-108-*-*")]

Note : "/var/log/steps/$rec_prod_step_function$/stdout"

Thanks in advance.

Labels (3)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Rather than append, try using map

index="os" (source="/var/log/steps/*/controller") sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-108-*-*") |
transaction source startswith=("/code/ttt_env.sh" OR "/code/ttt_gen.sh") endswith="startRun() called" |
rex field=_raw "(?<rec_prod_step_function>\bs-[a-zA-Z0-9_]+)" | where rec_prod_step_function="*"
| map search="search index=\"os\" source=\"/var/log/steps/$rec_prod_step_function$/stdout\" sourcetype=\"too_small\" (host=\"ip-101-108-*-*\" OR host=\"ip-101-108-*-*\")" maxsearches=0
0 Karma

alok
Loves-to-Learn Everything

I ran the suggested query getting a error message 

Error in 'map': Did not find value for required attribute 'rec_prod_step_function'.

Please suggest.

As debug I break the query when I ran 

 

index="os" (source="/var/log/steps/*/controller") sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-108-*-*") |
transaction source startswith=("/code/ttt_env.sh" OR "/code/ttt_gen.sh") endswith="startRun() called" |
rex field=_raw "(?<rec_prod_step_function>\bs-[a-zA-Z0-9_]+)" | where rec_prod_step_function="*"

 

It is not returning any event.

but when I used "where" to "search"

 

index="os" (source="/var/log/steps/*/controller") sourcetype="too_small" (host="ip-101-108-*-*" OR host="ip-101-108-*-*") |
transaction source startswith=("/code/ttt_env.sh" OR "/code/ttt_gen.sh") endswith="startRun() called" |
rex field=_raw "(?<rec_prod_step_function>\bs-[a-zA-Z0-9_]+)" | search rec_prod_step_function="*"

 

Query returns two events that is correct.

Please suggest.

Thanks !!

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

If you get the right results with search instead of where, does the map function do what you want?

I don't understand why search works but where doesn't. Does the rec_prod_step_function field get extracted successfully? Can you provide the results of the successful query?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...