Convert string to command, for dynamic union search

jlr — Mon, 16 Sep 2019 03:26:54 GMT

TL;DR - Is there a way (without custom scripts or commands) to run a command from a string in the format of a union that contains a dynamic number of subsearches?

I have quite a few heavy dashboards that get hit by many users, quite frequently. In order to save both processing power and user time, I've been caching search results (such as in a loadjob or outputlookup) and stitching them together dynamically. These dashboards all use a single savedsearch, and an instance of the search will cache data hourly, usually saving a day's worth of back-data. The dashboards then take the saved data and combine it with a call of the same search from the start of the hour until now(), and it's significantly faster.

I'm trying to create a macro that can do this intelligently so I don't have to re-build it per dashboard/savedsearch etc. I've created a search that can generate a string that, when used as a command, searches fine. However, I've been unable to find a way to get it to actually run - as far as I or anyone in my team can tell, anything that's a string can only be run as a search, which is insufficient for my needs as it ends up literally searching for a string talking about unions.

The example string that's being generated by the query below reads nicely as: (though NB, the output string could just as easily be a single line with no union or sort, or a union of two subsearches)

| union
  [ | savedsearch my_savedsearch earliest=1568343703.000000 latest=1568462400.000000 ]
  [ | inputlookup mycsv.csv  ]
  [ | savedsearch my_savedsearch earliest=1568602800.000000 latest=1568602843.000000 ]
| sort - _time

The generating query has an initial eval statement that I intended to use when moving it into a macro, which is stubbing the various args. Modifying these will dynamically change the number of subsearches/times (as the user might be requesting a time period relative to the cached data that is before, after, during, or some combination of each). Variants of this have been attempted using things like append, appendpipe, multisearch, but all seem to fail due to various reasons except the above (circular dependencies, streaming commands etc). The string generated, when run against a valid dataset, works. When the command is passed as the search itself using subsearches/return etc all fail - as far as we can tell, it's trying to run it as a search, or otherwise is unable to handle it.

Generating query: (replace contents in the initial eval for different dataset examples)

| makeresults
| eval
  timerange_string="| inputlookup mycsv.csv",
  timerange_earliest=relative_time(now(), "-1d@d"),
  timerange_latest=relative_time(now(), "@h"),
  time_string="| savedsearch my_savedsearch",
  time_earliest=relative_time(now(), "-3d"),
  time_latest=relative_time(now(), "-1m")
| eval
  earliest_period=case(
    time_earliest < timerange_earliest,-1,
    (timerange_earliest <= time_earliest AND time_earliest <= timerange_latest),0,
    timerange_latest < time_earliest,1,
    true()=true(),"null"
  ),
  latest_period=case(
    time_latest < timerange_earliest,-1,
    (timerange_earliest <= time_latest AND time_latest <= timerange_latest),0,
    timerange_latest < time_latest,1,
    true()=true(),"null"
  )
| fields - _time
| eval
  union_string=if(earliest_period!=latest_period,"| union [ ",""),
  search=
     if(earliest_period=0 OR latest_period=0,"| search","")
    .if(earliest_period=0," earliest=".time_earliest,"")
    .if(latest_period=0," latest=".time_latest,""),
  inputlookup=timerange_string." ".search,
  inputlookup_string=if(earliest_period*latest_period<1,inputlookup,""),
  ss1=time_string." earliest=".time_earliest." latest=".if(latest_period=-1,time_latest,timerange_earliest),
  ss1_string=if(earliest_period=-1,ss1,""),
  ss2=time_string." earliest=".if(earliest_period=1,time_earliest,timerange_latest)." latest=".time_latest,
  ss2_string=if(latest_period=1,ss2,""),
  sort_string=if(earliest_period!=latest_period," ] | sort - _time","")
| eval search=union_string.ss1_string.if(earliest_period=-1 AND latest_period!=-1," ] [ ","").inputlookup_string.if(latest_period=1 AND earliest_period!=1," ] [ ","").ss2_string.sort_string
| fields search

There are some similar questions (don't have enough karma to post links), but none directly tackle this use case, as map doesn't seem to be a viable solution here (triggers circular dependencies) and the general advice given with return doesn't work (as mentioned above).

Is there an obvious option I'm missing here? The next step I'd be looking at would be writing a custom command, which is less portable than a macro. I'll be working on that at some point, but if there's a way to do this with a macro using a basic Splunk query, it'd be my preference.

Re: Convert string to command, for dynamic union search

thomasroulet — Mon, 16 Sep 2019 14:35:38 GMT

Hello,

a little bit ugly but it could work :

`your_SPL`
| map search="|makeresults 1 | map search=$search$"

your savedsearch has to have 2 arguments :

earliest
latest

==> example: index=_* earliest=$earliest$ latest=$latest$

Re: Convert string to command, for dynamic union search

jlr — Tue, 17 Sep 2019 04:17:17 GMT

While this does function, it causes the search to be subject to the subsearch limit, e.g.

info : [map]: [map]: Search Processor: Subsearch produced 76673 results, truncating to maxout 10000.

This doesn't occur when run as the raw command. The only way around this (that I know of) would be global to my org, so not an option. (The above number is the search results at the moment for 1 day's worth of data)

Thanks for the suggestion though, it was worth trying. I do find it interesting that it works only when you wrap the map inside another map though.

topic Re: Convert string to command, for dynamic union search in Splunk Search

Convert string to command, for dynamic union search

Re: Convert string to command, for dynamic union search

Re: Convert string to command, for dynamic union search