topic Re: How abort a search based on a condition? in Splunk Search

How abort a search based on a condition?

reed_kelly — Fri, 17 Mar 2017 17:20:40 GMT

I have a search that writes a lookup file at the end. I also have searches that end in a collect command. And there are other things that I would like to do that cause side-effects. What I am looking for is a way to abort a search before getting to the commands with side effects.

For example,

index=abc ...
...
|abort condition=[count < 50]
...
|collect index=summary
|outputlookup abc.csv
|my_custom_command_to_post_to_twitter
|etc...

I could probably do it all by wrapping the latter half in a map command, but I am looking for an easier way.

Re: How abort a search based on a condition?

woodcock — Sun, 19 Mar 2017 03:58:32 GMT

This explains how to do it with map but you can just as easily do it with a subsearch:

https://answers.splunk.com/answers/172541/is-it-possible-to-purposely-cause-a-scheduled-sear.html

Also, instead of having the search cause an error with bogus time values, you could just as easily replace the entire search with |noop.

Re: How abort a search based on a condition?

reed_kelly — Tue, 21 Mar 2017 16:03:31 GMT

I'd rather not do it with a map, because some of my finishing commands already leverage map. For the "Accept" how would you do it with a subsearch?

Re: How abort a search based on a condition?

somesoni2 — Tue, 21 Mar 2017 16:32:33 GMT

How are you running the search, scheduled or ad-hoc?

Re: How abort a search based on a condition?

reed_kelly — Tue, 21 Mar 2017 17:27:36 GMT

I'm running many scheduled searches. In SQL, there is the "on error" capability that let's you avoid taking further action if something broke before.

Many of our searches do something like the following:

|dbxquery connection=xxx query="select * from tablex"
|massage the data
|outputlookup tablex_fast_lookup

If the SQL command dies due to a SQL issue, then we end up writing an empty lookup file. Once the lookup file is empty, dozens of other things break. I suppose I could convert it to a kv-store and merge data to the lookup with a last update timestamp. Then we could purge older data based on the timestamp.

I figured that if I could abort the command, then the lookup file would have stale data, but it would not be empty.

Re: How abort a search based on a condition?

woodcock — Tue, 21 Mar 2017 18:31:19 GMT

Or you can do it with a subsearch, like this:

YOUR BASE SEARCH HERE [| noop 
| stats count AS blackout 
| addinfo 
| eval blackout=case((SomeLogic="For Blackout Here"), "YES",
    (OtherLogic="For Blackout Here"), "YES",
    true(),"NO") 
| eval earliestMaybe=if((blackout=="NO"), info_min_time, now()) 
| eval latestMaybe=if((blackout=="NO"), info_max_time, 0) 
| eval search="earliest=" . earliestMaybe . " latest=" . info_max_time]
| YOUR POST SEARCH HERE

Re: How abort a search based on a condition?

somesoni2 — Tue, 21 Mar 2017 18:45:03 GMT

For just this particular problem (empty lookup when no data), you could do something like this
Updated

 |dbxquery connection=xxx query="select * from tablex"
 |massage the data
 | eval type="new"  | append [ | inputlookup tablex_fast_lookup | eval type="old"]
 | eventstats count(eval(type="new")) as hasData | eval filter=if(hasData>0,"new","old") 
 | where type=filter | fields - type hasData filter
 |outputlookup tablex_fast_lookup

Basically, if base search has no data, re-add the existing data from lookup again.

Re: How abort a search based on a condition?

reed_kelly — Tue, 21 Mar 2017 19:43:16 GMT

For the append, I think you mean "inputlookup". I know what you mean and this works well for that case.
-Thanks

Re: How abort a search based on a condition?

reed_kelly — Tue, 21 Mar 2017 19:59:58 GMT

Taking it one step further, I made a global macro:

[outputlookup_if_data(1)]
args = lookup_file
definition = eval upx_type="new" | append [ |inputlookup $lookup_file$ | eval u
px_type="old"] | eventstats count(eval(upx_type="new")) as upx_hasData | eval u
px_filter=if(upx_hasData>0,"new","old") | where upx_type=upx_filter | fields -
upx_type upx_hasData upx_filter | outputlookup $lookup_file$

Re: How abort a search based on a condition?

mmol — Wed, 30 Sep 2020 04:10:30 GMT

You could make use of the override_if_empty attribute of outputlookup:

 | dbxquery connection=xxx query="select * from tablex"
 |massage the data
 |discard all events when an error condition occurs, maybe using:
 | eventstats count as dbresults
 | where dbresults > 10000 
 | outputlookup tablex_fast_lookup override_if_empty=false