Summary indexing

[deepak02]

Hi,

I am very new to summary indexing. I need advice on what to set the interval to.

My dashboard refreshes every five minutes, and runs complicated queries (with regex, lots of stats etc.) over the last 15 mins.

How often should I run the summary index (every minute??), and what is the logic behind it?

Can I include regex as part of the summary index?

Thanks,
Deepak

[somesoni2]

The interval of summary index should be based on the time range it uses. So if it looks over last 15 mins, it should run at 15 min interval so that there is no overlap or gaps between the data being summarized. For best practices, use the cron schedule option with allowing some time to account for data ingestion delay. So your search time range could be earliest=-3m@m latest=-18m@m with cron schedule = 3-59/15 * * * *, allowing a 3 min buffer for data ingestion to be completed.

[deepak02]

Thankyou.
Just to confirm....

If my search runs at 12.20 over a 15 minute interval
(i.e.)
Search runs at: 12.20
Search time range: 12.05 to 12.20

Summary index runs at: 12:18
Summary index earliest: 12.03
Summary index latest: 12.18

Will the search running at 12.20 correctly use only the events between 12.05 to 12.20 even if the summary index is for the chunk 12.03 to 12.18?

[somesoni2]

There is small delay between data being ingested and become searchable (so that it appears in result of summary index search. That's why we add some additional delay. The summary index search running at 12:18 would summarize data from 12:00 to 12:15 (we added 3 min delay so that data for this period is searchable). So if you're running a search at 12:20, it should look at data for 12:00 to 12:15 (earliest=-5m@m lastest=-20m@m) for correct results.