topic Re: Is my map search being parsed before variable substitution? in Splunk Search

Is my map search being parsed before variable substitution?

mlf — Tue, 29 Sep 2020 08:18:15 GMT

Having issues with the following:

| map search="search index=summary search_name=\"$summary_search$\" $summary_selector$"

It returns the expected results when $summary_selector$ evaluates to a simple string, "Foo" or "Bar=Foo", but silently fails in cases where it contains a space, or multiple space-separated terms, "Bar=Foo " or "Bar=Foo Baz=xxx". This leads me to believe that the map search is being parsed before variable substitution and $summary_selector$ is being forced to a string literal. I.e. it seems the search is evaluated as:

search index=summary search_name="Mysearch" "Bar=Foo Baz=xxx"

rather than the intended:

search index=summary search_name="Mysearch" Bar=Foo Baz=xxx

Can anyone confirm the parsing/substitution order or suggest a workaround?

Re: Is my map search being parsed before variable substitution?

jplumsdaine22 — Mon, 04 Jan 2016 14:38:51 GMT

Are those variables field names from the previous search command? With Map $field$ should evaluate to the value for field for each input to the map command.

So if you summary_selector is a multivalue field, then split it first and then change your map. Something like this:

rex yourMVfield="Foo=(?<Foo>foo?)Baz=(?<Baz>baz?)"  | map search="search index=summary search_name=$summary_search$ Foo=$Foo$ Baz=$Baz$"

See 'Usage' at http://docs.splunk.com/Documentation/Splunk/6.3.2/SearchReference/Map

Also, theres a good chance you don't need map at all depending on what your base search is.

Re: Is my map search being parsed before variable substitution?

mlf — Tue, 29 Sep 2020 08:18:18 GMT

$summary_search$ would be the name of a populating search. $summary_selector$ would be part of a search string used to select data from the summary for forecasting. The use case is something like this:

| localop
| rest /services/saved/searches 
| search title="XYZ - Predict - ATM volume" 
| rex field=search "search_name=\"(?<summary_search>[^\"]+)\" (?<summary_selector>[^\|]+)" 
| rename title as forecast_search 
| map search="search index=summary search_name="$forecast_search$" OR (search_name="$summary_search$" $summary_selector$) 
   | timechart span=10m sum(low) as low, sum(pred) as forecast, sum(high) as high, sum(count) as actual "

where the "* - Predict - *" search is of the form:

index=summary search_name="XYZ - Summary - Count by Type and Action" MessageType=Atm
| timechart span=10m sum(count) as count
| `predict5w(count, 90, +1d, 1)`

The basic idea is a form that lets the user select a forecasting (* - Predict - *) search, gets the underlying summary search and selection criteria, and displays the actual (summery) vs forecast data for that metric. In this example, summary_selector="MessageType=Atm " so the search fails. If I adjust the regex so summary_selector="MessageType=Atm" it works, but only because MessageType=Atm is functionally equivalent to "MessageType=Atm" for this particular search. In the case where summary_selector="MessageType=Atm Action=Denied", the search would fail since "MessageType=Atm Action=Denied" is not equivalent to MessageType=Atm Action=Denied.

I figured out one possible workaround using a where/searchmatch clause in the map search:

| localop 
| rest /services/saved/searches 
| search title="XYZ - Predict - ATM volume" 
| rex field=search "search_name=\"(?<summary_search>[^\"]+)\" (?<summary_selector>[^\|]+)" 
| rename title as forecast_search
| eval filter="search_name=\"".forecast_search."\" OR (search_name=\"".summary_search."\" ".summary_selector.")"
| map search="search index=summary search_name=\"$forecast_search$\" OR search_name=\"$summary_search$\" | where searchmatch($filter$) | timechart span=10m sum(low) as low, sum(pred) as forecast, sum(high) as high, sum(count) as actual "

I'm still checking edge cases, but it seems to work so far. Not as efficient as it could be though.

Re: Is my map search being parsed before variable substitution?

jplumsdaine22 — Mon, 04 Jan 2016 18:02:31 GMT

OK I see.

Well you can tokenize the search with the following logic

| localop 
| rest /services/saved/searches 
| search title="XYZ - Predict - ATM volume"\
| makemv delim=" " search
| table search 
| mvexpand search 
| rex field=search "(?<key>\w+?)=(?<value>.*)" 
| eval {key}=value 
| fields - key value search 
| stats values(*) as *

The problem is it will be hard to use that with map. I would try using a subsearch instead