How can I prevent adding duplicate events when indexing via scheduled reports to a summary index?

[adamblock2]

I currently have a saved search configured to write its results to a summary index. This search is scheduled to run at 02:00 every night. The search starts with the following code: starthoursago=26 endhoursago=2 index=mail .... It is my understanding that the starthoursago/endhoursago commands set the time range of the search to 00:00 - 24:00 of the previous day.

I am thinking that there must be a better way to schedule this search so to avoid writing duplicate events to the summary index. If possible, I would like the search to run more frequently - every hour if possible.

Assistance with this will be greatly appreciated.

Thank you.

[ddrillic]

Based on the recent discussion at How can we avoid data loss in the summary indexes when there is an indexing latency in the cluster?

We ended up with something like -

It covers an hour of data, and back two to three hours ago - we like it ; -)