Splunk Search

Can I use a Lookup to control the order Saved Searches are run in?

gjlewis
Explorer

Hi,

We have 10 Saved Searches that we want to schedule to run in chronological order. We can do this using cron, but it is important that each search doesn't start until the previous search completes. This is because these searches output to the same lookup files, so we don't want to end up with different saved searches trying to update the same file at the same time.

In order to mitigate this scenario, I have been trying to create a simple control lookup table, which consists of a single column 'SS' with a single variable starting with '1', this will then update with a value of 1 through to 10.

When saved search 1 starts I want it to check the control lookup table for the value '1'. If it is '1' the search will run and update the output lookup file, otherwise, if the value isn't '1' the search won't run, or it can run but it won't find any events to update as the saved search value didn't match. Once the search has completed the control lookup table will update to the next saved search number (this part I can do quite easily with eval and outputlookup).

The problem I have is getting the control value passed into the main search, so far it won't recognise it. This is what I have tried:

index=email sourcetype=emailexport 
| inputlookup append=t scheduleLookup.csv 
| eval run=if(SS=1,"yes","no") 
| search run=yes 
| table _raw

This returns nothing, however, if I change the search to run=no, the event is found. The actual search does more data manipulation with evals than the above, but I changed it to table _raw to hopefully simplify the problem.

I'm sure I'm missing something simple, but any advice would be appreciated.

Thanks

0 Karma
1 Solution

nickhills
Ultra Champion

The problem is that Splunk is evaluating your 'if' against every event in your search, and its will only match once when the lookup is loaded, ie on one event (which wont have any _raw) try |table _raw SS run to see what I mean.

You should be able to 'bring' the SS value into the rest of you data set with something like this:

index=email sourcetype=emailexport
 | inputlookup append=t scheduleLookup.csv 
 | eventstats values(SS) as SSVal
 | eval run=if(SSVal=1,"yes","no") 
 | search run=yes
 | table _raw SSVal run
If my comment helps, please give it a thumbs up!

View solution in original post

0 Karma

nickhills
Ultra Champion

The problem is that Splunk is evaluating your 'if' against every event in your search, and its will only match once when the lookup is loaded, ie on one event (which wont have any _raw) try |table _raw SS run to see what I mean.

You should be able to 'bring' the SS value into the rest of you data set with something like this:

index=email sourcetype=emailexport
 | inputlookup append=t scheduleLookup.csv 
 | eventstats values(SS) as SSVal
 | eval run=if(SSVal=1,"yes","no") 
 | search run=yes
 | table _raw SSVal run
If my comment helps, please give it a thumbs up!
0 Karma

gjlewis
Explorer

Perfect - thank you!

0 Karma
Get Updates on the Splunk Community!

New Cloud Intrusion Detection System Add-on for Splunk

In July 2022 Splunk released the Cloud IDS add-on which expanded Splunk capabilities in security and data ...

Happy CX Day to our Community Superheroes!

Happy 10th Birthday CX Day!What is CX Day? It’s a global celebration recognizing innovation and success in the ...

Check out This Month’s Brand new Splunk Lantern Articles

Splunk Lantern is a customer success center providing advice from Splunk experts on valuable data insights, ...