Knowledge Management

How to schedule summary searches with "dependencies"?

arkadyz1
Builder

I'm trying to create the following hierarchical structure of the summary searches:
1st search:

... | collect index=summary marker="search_type=type1"

2nd search:

... | collect index=summary marker="search_type=type2"

3rd search:

index=summary search_type=type2 | ... | join type=inner commonField [ search index=summary search_type=type1 ] | ...

I'd like to schedule all three to run every 15 minutes. Naturally, I'd like the first two to finish running before the third one even starts. Is there a way to achieve that?

One of the ideas I had was this: schedule the first two (independent) searches at :00, :15, :30, :45 and set the time range for them as "-15m@m" and "@m", while the third one will be scheduled at :01, :16, :31, :46 (hope cron supports that) and will have the range of "-16m@m" through "-1m@m". This does not necessarily make them dependent but gives me some chances.

Is there a better way of achieving my goal?

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

You can create such a cron schedule two ways:

1-59/15 * * * *
1,16,31,46 * * * *

Both do the same thing, running your search one minute after every quarter hour.

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

You can create such a cron schedule two ways:

1-59/15 * * * *
1,16,31,46 * * * *

Both do the same thing, running your search one minute after every quarter hour.

martin_mueller
SplunkTrust
SplunkTrust

@cpetterborg: My two schedules intentionally do the same thing, I'm showing both ways of specifying the same 1m-offset-15m-interval runs.

I personally consider the 1-59/15 notation to be more maintenance friendly than 1,16,41,46. So easy to make and miss a small typo there that royally screws up your scheduling, alerting, summary indexing, etc.
Also, if you're doing five-minute intervals specifying twelve numbers becomes quite tedious. Shifting them by a minute would mean you have to change all twelve, with 1-59/5 you only have to change the 1.

markthompson
Builder

I agree with @martin_mueller It's better that @arkadyz1 gets into the habit of making sure his Splunk is maintenance friendly so that others can understand how it works, as well as making any edits easier for colleagues who might need to change it without contacting him.

@cpetterborg your solution does work as well, but it's not as friendly to any big company as stated by Martin, the easier it is to change, the more appealing it is.

arkadyz1
Builder

Thanks to all who replied! I'm also in the camp of maintainability, so 1-59/15 seems like a preferred solution to me.

cpetterborg
SplunkTrust
SplunkTrust

Better:

0,15,30,45 * * * *
1,16,31,46 * * * *

The 1-59 will do the first of the first at 1, which would collide with the first of the second at 1. You could also do:

0-59/15 * * * *

for the first one, but I think the first is really more detailed and easier to understand how they relate.

0 Karma

arkadyz1
Builder

Great, saved me time and doubts! Thanks!

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...