Knowledge Management

How to schedule summary searches with "dependencies"?

Builder

I'm trying to create the following hierarchical structure of the summary searches:
1st search:

... | collect index=summary marker="search_type=type1"

2nd search:

... | collect index=summary marker="search_type=type2"

3rd search:

index=summary search_type=type2 | ... | join type=inner commonField [ search index=summary search_type=type1 ] | ...

I'd like to schedule all three to run every 15 minutes. Naturally, I'd like the first two to finish running before the third one even starts. Is there a way to achieve that?

One of the ideas I had was this: schedule the first two (independent) searches at :00, :15, :30, :45 and set the time range for them as "-15m@m" and "@m", while the third one will be scheduled at :01, :16, :31, :46 (hope cron supports that) and will have the range of "-16m@m" through "-1m@m". This does not necessarily make them dependent but gives me some chances.

Is there a better way of achieving my goal?

0 Karma
1 Solution

SplunkTrust
SplunkTrust

You can create such a cron schedule two ways:

1-59/15 * * * *
1,16,31,46 * * * *

Both do the same thing, running your search one minute after every quarter hour.

View solution in original post

SplunkTrust
SplunkTrust

You can create such a cron schedule two ways:

1-59/15 * * * *
1,16,31,46 * * * *

Both do the same thing, running your search one minute after every quarter hour.

View solution in original post

SplunkTrust
SplunkTrust

@cpetterborg: My two schedules intentionally do the same thing, I'm showing both ways of specifying the same 1m-offset-15m-interval runs.

I personally consider the 1-59/15 notation to be more maintenance friendly than 1,16,41,46. So easy to make and miss a small typo there that royally screws up your scheduling, alerting, summary indexing, etc.
Also, if you're doing five-minute intervals specifying twelve numbers becomes quite tedious. Shifting them by a minute would mean you have to change all twelve, with 1-59/5 you only have to change the 1.

Builder

I agree with @martin_mueller It's better that @arkadyz1 gets into the habit of making sure his Splunk is maintenance friendly so that others can understand how it works, as well as making any edits easier for colleagues who might need to change it without contacting him.

@cpetterborg your solution does work as well, but it's not as friendly to any big company as stated by Martin, the easier it is to change, the more appealing it is.

Builder

Thanks to all who replied! I'm also in the camp of maintainability, so 1-59/15 seems like a preferred solution to me.

SplunkTrust
SplunkTrust

Better:

0,15,30,45 * * * *
1,16,31,46 * * * *

The 1-59 will do the first of the first at 1, which would collide with the first of the second at 1. You could also do:

0-59/15 * * * *

for the first one, but I think the first is really more detailed and easier to understand how they relate.

0 Karma

Builder

Great, saved me time and doubts! Thanks!

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!