topic Re: Automate backfill script in Splunk Search

Automate backfill script

mwdbhyat — Fri, 19 Aug 2016 11:14:59 GMT

Hi,

I need to automate the backfill script for about 60 searches.. Is there a way to put all 60 searches in a single script and then make them run one by one(wait until previous one is done before continuing to next)

Thanks

Re: Automate backfill script

skoelpin — Fri, 19 Aug 2016 11:19:37 GMT

You could have the backfill command run then have it produce an exit code when it's complete which will then trigger the next backfill to begin

Re: Automate backfill script

mwdbhyat — Tue, 29 Sep 2020 10:39:39 GMT

Thanks - noob question but could you give me an example of how this would work?

starting script code ?

./splunk cmd python fill_summary_index.py -app APPNAME -name SEARCHNAME -et -12w@w -lt now -dedup true

exit script code ?

Re: Automate backfill script

Raghav2384 — Tue, 29 Sep 2020 10:42:14 GMT

Hey @mwdbhyat,

fill_summary_index.py takes an argument called -j <integer> (max 😎 this will decide the concurrency. If not specified , one search backfill is executed at a time. As far as running automated for 60 searches, are they all in a same app? Can you move them to xyz app (just 60 and nothing more) and use -app xyz -name * or get creative pragmatically where you have an array and run it through some sort of controlled loop.

What i feel tricky is the time frame you need to select if all these 60 have different windows. I do the backfills quite often and i have wrapped them in shell scripts and tied them up to cron based on their original schedules (offcourse -dedup true)

Hope this helps!

Thanks,
Raghav

Re: Automate backfill script

mwdbhyat — Fri, 19 Aug 2016 11:49:07 GMT

Hi Raghav,

Thanks for the reply.. It is for 1 app yes. Basically I have created a txt file with all 60 searches listed with the ./backfill command as above. Initially just to add in previous 12w worth of data I just want it to do the automated backfill once. So if I just run my txt file with 60 searches listed, will it just do them 1 at a time until the whole list is done ?

Re: Automate backfill script

skoelpin — Fri, 19 Aug 2016 11:54:35 GMT

Assuming that you're working on a Linux machine..

http://bencane.com/2014/09/02/understanding-exit-codes-and-how-to-use-them-in-bash-scripts/

This example will require you to create multiple scripts and one script will trigger the next script after it completes.. Alternatively if you want a single script, you could use a WAIT for input and have the script wait until the backfill is complete which will start the next backfill..

#!/bin/bash

./splunk cmd python fill_summary_index.py -app APPNAME -name SEARCHNAME -et -12w@w -lt now -dedup   true

if [ $? -eq 0 ]
then
  echo "Successfully ran backfill"
  exit 0
else
  echo "Errors running backfill" >&2
  exit 1
fi

Here's another method of running which will look at the process ID and execute on a loop until all the backfills run

pid=$(ps -opid= -C your_script_name)
while [ -d /proc/$pid ] ; do
    sleep 1
done && ./your_other_script

Re: Automate backfill script

Raghav2384 — Tue, 29 Sep 2020 10:42:17 GMT

Correct, keeping the volume of searches aside, that script fill_summary_index.py is designed to execute one search at a time unless asked it to do more.

example:
./splunk cmd python fill_summary_index.py -app xyz -name * -dedup true -showprogress true -et -7d -lt now -j 8 -owner admin
will run / try to run 8 searches at a time (Whole different story if you do not have 8 cores on you search head....rule is 1 search per core).
& if you just not mention -j argument, it will run one search , waits for it to finish and then move on to the next.

Hope this helps!

Thanks,
Raghav

Re: Automate backfill script

mwdbhyat — Fri, 19 Aug 2016 11:56:13 GMT

Sweet, thanks a lot!

Re: Automate backfill script

mwdbhyat — Fri, 19 Aug 2016 11:56:51 GMT

Awesome, thank you!

Re: Automate backfill script

mwdbhyat — Fri, 19 Aug 2016 12:05:42 GMT

How would I pass the admin username in just once with creds as it is asking me for each search to type in the username and pass?

Re: Automate backfill script

Raghav2384 — Fri, 19 Aug 2016 12:14:55 GMT

wrap that in a shell script and pass it one time. That way you do not have to type it every time.

-auth admin:xxxx (Please remove it as soon as your backfill is complete).

Re: Automate backfill script

mwdbhyat — Fri, 19 Aug 2016 12:18:21 GMT

Cool, thanks!