Can I use a Lookup to control the order Saved Searches are run in?

[gjlewis]

Hi,

We have 10 Saved Searches that we want to schedule to run in chronological order. We can do this using cron, but it is important that each search doesn't start until the previous search completes. This is because these searches output to the same lookup files, so we don't want to end up with different saved searches trying to update the same file at the same time.

In order to mitigate this scenario, I have been trying to create a simple control lookup table, which consists of a single column 'SS' with a single variable starting with '1', this will then update with a value of 1 through to 10.

When saved search 1 starts I want it to check the control lookup table for the value '1'. If it is '1' the search will run and update the output lookup file, otherwise, if the value isn't '1' the search won't run, or it can run but it won't find any events to update as the saved search value didn't match. Once the search has completed the control lookup table will update to the next saved search number (this part I can do quite easily with eval and outputlookup).

The problem I have is getting the control value passed into the main search, so far it won't recognise it. This is what I have tried:

index=email sourcetype=emailexport 
| inputlookup append=t scheduleLookup.csv 
| eval run=if(SS=1,"yes","no") 
| search run=yes 
| table _raw

This returns nothing, however, if I change the search to run=no, the event is found. The actual search does more data manipulation with evals than the above, but I changed it to table _raw to hopefully simplify the problem.

I'm sure I'm missing something simple, but any advice would be appreciated.

Thanks

[nickhills · Accepted solution]

The problem is that Splunk is evaluating your 'if' against every event in your search, and its will only match once when the lookup is loaded, ie on one event (which wont have any _raw) try |table _raw SS run to see what I mean.

You should be able to 'bring' the SS value into the rest of you data set with something like this:

index=email sourcetype=emailexport
 | inputlookup append=t scheduleLookup.csv 
 | eventstats values(SS) as SSVal
 | eval run=if(SSVal=1,"yes","no") 
 | search run=yes
 | table _raw SSVal run

[gjlewis]

Perfect - thank you!