How to backfill a summary index with a restricted time for each day?

[jyamie]

I would like to backfill my index up by 2 months. The query however, is time sensitive and requires the day span to be only between 7am-9pm. Currently, my only method is to manually change the earliest and latest times in both the search and the summary index settings to earliest=-1d@d+7h latest=-1d@d+20h, and then to earliest=-2d@d+7h latest=-2d@d+20h, etc. etc.. you can see just how tedious and time-consuming this can become.

Is there any way that I would not have to insert any relative day into my period, to be able to run my overall index search for 30 days with days only involving data between 7am-9pm everyday? if there were an earliest=7h latest=20h kind of deal, that would be great, but I have not found any yet.
Thanks in advance

[woodcock]

You do know about the backfill command, right? You should be able use your standard daily SI-populating search as-is as described here:

http://docs.splunk.com/Documentation/Splunk/6.2.5/Knowledge/Managesummaryindexgapsandoverlaps

[jyamie]

actually i didn't know about the backfill command. bit of a splunk noob here. thanks!

[somesoni2 · Accepted solution]

You can do something like this

1) Create a scheduled saved search for your query. This query should run daily and summarize data for yesterday
2) For the query, add earliest=-1d@d+7h latest=-1d@d+20h (within the query)
3) For saved search's Start time use -1d@d, for Finish time, use @d (this just to enable scheduling, search'es time range is overridded in the search itself in step 2)
4) For cron use something like this 45 6 * * * (daily once at 6:45 AM)
5) Use the Splunk backfill utility to backfill any duration at once.

Sample command:

Search Time Range= -1d@d to @d
Since, we summarize yesterday's data, to backfill for Jan-01,2015 to Jan-31-2015, the search should run from Jan-02 (to summarize jan-01) to Feb-01 (to summarize Jan-31)

Backfill job Scheduling time
Earliest Time:01/02/2015 00:00
In Epoch: 1420174800  -- to be used as argument et

Latest Time: 02/01/2015 23:59:59.000000
In Epoch: 1422853199   - to be used as argument lt

Backfill Unix Command (after cd $Splunk_home/bin)
./splunk cmd python fill_summary_index.py -app YourAppName -name "YourSearchName" -et 1420174800  -lt 1422853199   -j 1 -dedup true -auth AdminUserName:AdminPasswordName &