Hi,
I have a pretty long search I want to be able to utilize as a savedsearch and allow others benefit from one shared search and maybe mutually edit the search, if need be. There is a part in the search utilizing a structure
search index=ix2 eventStatus="Successful"
| localize timeafter=0m timebefore=1m
| map search="search index=ix1 starttimeu=$starttime$ endtimeu=$endtime$ (
[ search index=ix2 eventStatus="Successful"
| return 1000 eventID ] )
| stats values(client) values(port) values(target) by eventID
This is a simplified extraction of what I am really doing, but the search works fine when run as a plain direct search from the GUI. If I save it and try using it with
|savedsearch "my-savedsearch"
I get the error
Error in 'savedsearch' command: Encountered the following error while building a search for saved search 'my-savedsearch': Error while replacing variable name='starttime'. Could not find variable in the argument map.
It looks like the $starttime$ and $endtime$ cause trouble, but what can I do to come around? I want to have this stuff in a saved search to avoid operating with a long search all the time in the browser. also, it is essential to use the localize - map construction, because otherwise I am not able to run this search for long time windows and I would really like to be able to do it.
There was a ticket by @neerajs_81 about pretty much the same issue, but there were no details about the saved search and above all, there seemed not to be a solution.
The error is exactly what it says: The interpreter cannot determine values of placeholder tokens $starttime$ and $endtime$ without replace_me arguments. There absolutely are details about savedsearch command. From opening paragraph in savedsearch:
If the search contains replacement placeholder terms, such as $replace_me$, the search processor replaces the placeholders with the strings you specify. For example:
| savedsearch mysearch replace_me="value"
Before showing a possible fix, I want to warn against using map command against time. It is unclear what starttimeu and endtimeu are. Are these in raw events in index=ix1? Fundamentally, map is often not the best solution to any given problem. Try these two searches, one uses map, one doesn't:
| makeresults format=csv data="field1v, field2v
aaa, bbb"
| map field1v field2v search="| makeresults format=csv data=\"field1, field2, field3
abc, def, 1
aaa, bbb, 2
xxx, yyy, 3\" | search field1 = $field1v$ field2 = $field2v$"
| makeresults format=csv data="field1, field2, field3
abc, def, 1
aaa, bbb, 2
xxx, yyy, 3"
| search
[makeresults format=csv data="field1v, field2v
aaa, bbb"
| rename field1v as field1, field2v as field2]
The output is exactly the same, but the second one is easier to understand and easier to construct.
To apply to your problem, if the fields I questioned above all exist in raw events, your search would be better constructed as
search index=ix1
[search index=ix2 eventStatus="Successful"
| return 1000 eventID ]
[search index=ix2 eventStatus="Successful"
| localize timeafter=0m timebefore=1m
| fields starttime endtime
| rename starttime as startimeu, endtime as endtimeu]
| stats values(client) values(port) values(target) by eventID
Not only does this search not need replacement token, but also it is easier to maintain.
In short, to map? Or not to map? That is the question. $xxx$ in saved search is interpreted as replacement tokens whose values must be specified in the invocation command. This contradicts with the map command's use of $xxx$. You can say not differentiating value tokens used in different contexts is a design weakness in SPL. But that's what SPL has today. This is not a bug per se.
The error is exactly what it says: The interpreter cannot determine values of placeholder tokens $starttime$ and $endtime$ without replace_me arguments. There absolutely are details about savedsearch command. From opening paragraph in savedsearch:
If the search contains replacement placeholder terms, such as $replace_me$, the search processor replaces the placeholders with the strings you specify. For example:
| savedsearch mysearch replace_me="value"
Before showing a possible fix, I want to warn against using map command against time. It is unclear what starttimeu and endtimeu are. Are these in raw events in index=ix1? Fundamentally, map is often not the best solution to any given problem. Try these two searches, one uses map, one doesn't:
| makeresults format=csv data="field1v, field2v
aaa, bbb"
| map field1v field2v search="| makeresults format=csv data=\"field1, field2, field3
abc, def, 1
aaa, bbb, 2
xxx, yyy, 3\" | search field1 = $field1v$ field2 = $field2v$"
| makeresults format=csv data="field1, field2, field3
abc, def, 1
aaa, bbb, 2
xxx, yyy, 3"
| search
[makeresults format=csv data="field1v, field2v
aaa, bbb"
| rename field1v as field1, field2v as field2]
The output is exactly the same, but the second one is easier to understand and easier to construct.
To apply to your problem, if the fields I questioned above all exist in raw events, your search would be better constructed as
search index=ix1
[search index=ix2 eventStatus="Successful"
| return 1000 eventID ]
[search index=ix2 eventStatus="Successful"
| localize timeafter=0m timebefore=1m
| fields starttime endtime
| rename starttime as startimeu, endtime as endtimeu]
| stats values(client) values(port) values(target) by eventID
Not only does this search not need replacement token, but also it is easier to maintain.
In short, to map? Or not to map? That is the question. $xxx$ in saved search is interpreted as replacement tokens whose values must be specified in the invocation command. This contradicts with the map command's use of $xxx$. You can say not differentiating value tokens used in different contexts is a design weakness in SPL. But that's what SPL has today. This is not a bug per se.
Hi,
What I was trying to say about savedsearch details was a referring to the linked other post. In that post I don't think the details about the saved search are not disclosed.
About your reply, I couldn't make it work. I restructured my search, and as you might have guessed already, the real search is more complicated, but the point is the structure. Since you asked, all the data is so far in raw events. I could use
| map search="search earliest=$starttime$ latest=$endtime$ ...."
to achieve the same result
The essential part in the structure is to be able to search from idx1 with a narrow time window based on the timestamps from the events matched from idx2 obtained from a much wider span. The reason is simply that the link between the two indexes, eventID, is weak and also there is at present a lot more data in idx1. eventID is actually not guaranteed to be unique for any period of time, but with reasonable reliability it is unique for a short period of time. That is why I have used localize, and so far I have not been able to make localize work with anything but map. Moving the map to a separate subsearch ruined the search and it returned nothing.
I started thinking about what you suggested and created a construction like this:
search index=ix1
[search index=ix2 eventStatus="Successful"
| return 1000 eventID ]
[search index=ix2 eventStatus="Successful"
| eval Start=_time-60, End=_time, search="_time>".Start." AND _time<".End
| return 500 $search]
| stats values(client) values(port) values(target) by eventID
It seems to return what I want. My understanding is that the search will however search the whole globally defined time window for idx1 instead of only looking at the short periods of time we are interested in. I am not sure if that will actually have a huge effect on load it causes.
At the end of your reply you describe the root cause of the problem, namely the way SPL treats $xxx$ expansions. As you say, it is not a bug. It is a property or limitation of SPL.