Getting Data In

What is the best way to re-run a summary index to collect missed events after an outage?

Glasses
Builder

Hi - 

Let's say you have a scheduled query / report that runs daily (at mid-night) looking over a time range of Last 24 hours.  And you summarize the results to index=summary_index_foo.

There was a "foo" data source outage for a couple days, however you were able to backfill the data to index=foo.

What is the best to re-run the query without creating a lot of duplicates.   I am pretty sure if you use "collect" that will create duplicates.

But will re-scheduling a one-time clone of the report over the outage days and summarizing results create duplicates if the time range overlaps into the data (before and after the outage)?

In other words, the outage time frame was not to the minute, hour, or day exactly.  When you re-schedule/re-summarize the query will that create duplicates if the same data/event exists in the summary index for that time?   

Or will Splunk drop duplicates when using the summary index?    I am guessing duplicates will still be created but need a sanity check.

Thank you

 

Labels (1)
Tags (1)
0 Karma
1 Solution

gbansode
Explorer

@Glasses Do you want to backfill summary Index ? Here you go https://docs.splunk.com/Documentation/SplunkCloud/latest/Knowledge/Managesummaryindexgapsandoverlaps...

Run the fill_summary_index command from bin. Example below 
./splunk cmd python fill_summary_index.py -app is_app_one -name
"summary - count by user" -et -30d -lt now -j 8 -dedup true -auth
admin:changeme

View solution in original post

0 Karma

gbansode
Explorer

@Glasses Do you want to backfill summary Index ? Here you go https://docs.splunk.com/Documentation/SplunkCloud/latest/Knowledge/Managesummaryindexgapsandoverlaps...

Run the fill_summary_index command from bin. Example below 
./splunk cmd python fill_summary_index.py -app is_app_one -name
"summary - count by user" -et -30d -lt now -j 8 -dedup true -auth
admin:changeme
0 Karma

Glasses
Builder

TY, for the reply, I will try it.

 

 

0 Karma
Get Updates on the Splunk Community!

Is there an add-on for the Cisco Meraki devices?

We have many Cisco Meraki devices sending data via syslog to Splunk. Is there an add-on for ...

Should our Deployment Servers have the Search Head server role on them?

all of our stuff is on premcurrently our dedicated Deployment Servers also have the Search Head role on them, ...

Why am I unable to create dropdown static option that is "All" static options ...

Hi, I am trying to get a static option that is "All" the individual static options combined.  The mCode ...