topic Re: How to Re-Populate Summary Index? in Getting Data In

How to Re-Populate Summary Index?

skoelpin — Sat, 06 Feb 2016 03:38:15 GMT

I have a summary index set up to populate every hour. The forwarder that populated the summary index was down for a few days, after starting the forwarder, it grabbed the log files retroactively but did not send that data to the summary index. So how would I get that data from index=endeca to index=endeca-summary when the forwarder was down?

Re: How to Re-Populate Summary Index?

renjith_nair — Sat, 06 Feb 2016 04:56:12 GMT

Hi @skoelpin,

You can use fill_summary_index.py to back fill the data

Detailed instructions are available : http://docs.splunk.com/Documentation/Splunk/6.0.2/Knowledge/Managesummaryindexgapsandoverlaps

Re: How to Re-Populate Summary Index?

chimell — Mon, 08 Feb 2016 09:06:03 GMT

Hi
See this search code which use Collect command perhaps it will help you

index=endeca |collect index=endeca-summary

for more information about collect command see this link http://docs.splunk.com/Documentation/Splunk/6.3.3/SearchReference/Collect

Re: How to Re-Populate Summary Index?

skoelpin — Mon, 08 Feb 2016 21:34:43 GMT

Thanks, this is exactly what I'm trying to do, but running into a problem

I'm currently on my Splunk indexer and opened up powershell and navigated to Splunk\bin where the python script is and I'm entering the below command. After executing this, I'm getting back "Microsoft Windows Version 6.3 2013 Microsoft Corp. All rights Reversed"

cmd python fill_summary_index.py -index "endeca-summary"  -et 1449730800 -lt 1452898800  -dedup true -auth admin:xxxxxxx

*Summary index = endeca-summary
*Regular index = endeca
*I used epoch time for -et and -lt.. I want to test this out by running it for 1 hour in the middle of my missing data to confirm this works before attempting to backfill the month

Re: How to Re-Populate Summary Index?

somesoni2 — Mon, 08 Feb 2016 22:33:31 GMT

The summary index backfill works to re-run the scheduled search (which is populating the summary index) by simulating historical execution. The correct syntax (as mentioned in the link in the answer) is below.

splunk.exe cmd python fill_summary_index.py -app AppName -name "SearchName" -et epochEarliest -lt epochLatest -dedup true -auth -j N admin:changeme

Where, SearchName is the name of search populating your summary index and AppName is the app containing the search, epochEarliest and epochLatest are epoch time range for the search schedule (not the timerange used in search), N is number of parallel executions.

And since it requires the search name, it has to be run from Search Head where the search is available/enabled/scheduled.