SplunkJS/HTML Dashboards + map command + $foo$ substitution

[martin_mueller]

I'm trying to make a search involving the map command work from inside a SplunkJS/HTML Dashboard, ie launched by a SearchManager. Say this were my search in a regular search bar:

index=_internal error | localize | map search="search index=_internal earliest=$starttime$ latest=$endtime$"

$starttime$ and $endtime$ will be replaced by map based on the previous pipeline, that's working well.
Transferring this to AdvancedXML you'd need to double up the $ signs to escape them to get this:

index=_internal error | localize | map search="search index=_internal earliest=$$starttime$$ latest=$$endtime$$"

That works well too.
However, when I'm trying to run the search from a SearchManager, I can't get this to work. Running the second escaped query gives me no results and this warning:

Unable to run query 'search index=_internal earliest=$1390463531$ latest=$1390463736$'.

This suggests that my escaped $$starttime$$ was replaced while retaining the escaping $ signs. Removing the escaping $ signs gives me this warning:

Search query is not fully resolved.

In both cases I've set tokens to false to not substitute tokens from an input field or wherever.
Here's the SearchManager for completeness' sake:

var search1 = new SearchManager({
"id": "search1",
"status_buckets": 0,
"search": 'index=_internal error | localize | map search="search index=_internal earliest=$$starttime$$ latest=$$endtime$$"',
"earliest_time": "-15m@m",
"cancelOnUnload": true,
"latest_time": "now",
"app": utils.getCurrentApp(),
"auto_cancel": 90,
"preview": true
}, {tokens: false});

As expected, using mvc.tokenEscape(1) doesn't affect this, because token isn't set to true.

How do I get this to work?

[martin_mueller · Accepted solution]

I have found a workaround to make using the map command in SimpleXML and SplunkJS/Webframework dashboards work: Store the search run by map in a report rather than inline.

Old search, confusing SplunkJS with dollar signs:

index=_internal error | localize | map search="search index=_internal earliest=$starttime$ latest=$endtime$"

New search, not confusing SplunkJS anymore:

index=_internal error | localize | map my_saved_search

With a report called my_saved_search looking like this:

index=_internal earliest=$starttime$ latest=$endtime$

I can live with this 🙂

[martin_mueller]

This appears to work in 6.2.1 (maybe earlier?) without the saved search workaround.

[dfoster_splunk]

I was able to get this to work. See my code here: https://gist.github.com/davidfstr/1a5d8e9e7cd041f86a43

One possible complication is that the SearchManager's earliest_time wasn't being set to $starttime$.

Other notes:

• {tokens: false} is the default, so I omitted it.
• mvc.tokenSafe(...) is the same as {tokens: true} turned on for individual settings.

[martin_mueller]

The same issue is occurring with Splunk 6 SimpleXML dashboards. Consider this very simple example:

<dashboard>
  <row>
    <table>
      <searchString>index=_internal error | localize | map search="search index=_internal earliest=$starttime$ latest=$endtime$"</searchString>
    </table>
  </row>
</dashboard>

That yields "Search query is not fully resolved" as expected. Doubling up the dollar signs yields the same error message, and quadrupling the dollar signs gives me "Unable to run query 'search index=_internal earliest=$1394717566$ latest=$1394717753$'."

[martin_mueller]

Those quotes are required by the map command, running a working search in the regular searches view breaks if you remove the quotes. Without quotes everything after the first word is ignored, running the empty query "search" which must yield nothing.

Regardless I have tried your code snippet in the dashboard, and it doesn't produce any results while a working search in the searches view for the same timerange does.

Behind the scenes $$starttime$$ is still replaced with $1391068887$, but that isn't throwing an error because it's not part of the search string anymore.

[mgroves_splunk]

Hi Martin

Can you try this:

var search1 = new SearchManager({
"id": "search1",
"search": "index=_internal error | localize | map search=search index=_internal earliest=$$starttime$$ latest=$$endtime$$",
"status_buckets": 0,
"latest_time": "now",
"earliest_time": "-15m@m",
"cancelOnUnload": true,
"auto_cancel": 90,
"preview": true
}, {tokens: false});

I believe the quotes around the inner search for the map command is causing the problem and does not seem to be required.

-Mark

[martin_mueller]

Using $$token$$ and tokens:true or mvc.tokenSafe(1) also gives me "Search query is not fully resolved.", so SplunkJS is trying to substitute something where I don't want it to.

[martin_mueller]

Correct, I'm trying to get $token$ to appear literally in my search string. I have tried those two approaches already, as described in my question.

Using $token$ with tokens:false gives me an unresolved query error.

Using $$token$$ gives me a literal $$token$$, which then gets replaced with $12345$ if $token$'s value is 12345.
However, in this double-dollar case I think I may have had tokens still set to false... I'll set it to true tomorrow and give it another shot to see if it makes a difference.

[dfoster_splunk]

I am not familiar with the map command. Based on the doc it looks like you actually need to pass it literal dollar signs for it to do its own substitution. Therefore if you pass "$token$" with {tokens: false} it should get to the map command correctly. Or if you use "$ $token$ $" (no spaces) with {tokens: true} it should also work. I have not tested either of these.

[martin_mueller]

Do you have another idea to solve this?

[martin_mueller]

Yes, see http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/map for reference.

[dfoster_splunk]

get values from the pipeline before the map command into the search run by the map command

This appears to be an entirely different question. So you want to run a search, extract an earliest_time from it somehow, and run a "map" search command that takes the first search as input but uses the earliest_time that was extracted?

[martin_mueller]

...ran out of space...

The key here is to get values from the pipeline before the map command into the search run by the map command, not to get tokens from somewhere in the SplunkJS dashboard into the search.

[martin_mueller]

Thanks for taking a stab at this.
I've taken your SearchManager definition and ran it locally - while it does produce results, I fear they're incorrect.

Running the search from the SearchManager in the search app manually, I get events from the past ten-ish minutes, because _internal has no "error" events before that. However, running it from the SplunkJS dashboard gives me many more events including ones from up to 15 minutes ago.
My guess is that the "-15m@m" you set after creating the SearchManager are written into the map command rather than using the values from the localize.