Knowledge Management

How do I maintain quality of the summary index?

Explorer

You can you backfill to fill in missing pieces, but what happens when splunk or syslog run behind and events run partly late?

In this case you get a poor quality summary index. How can these be detected and updated?

Tags (1)

Splunk Employee
Splunk Employee

This is a difficult problem to solve. We're planning on automating this in a future version of Splunk, so that summarization on target searches is automatic and transparent.

In the meantime, our best suggestion is to monitor your inputs for lag, using the difference of _time and _indextime. If lag is detected or expected, you should purge records from your summary index using | delete and refill the gap in the summary.

We have some customers who will always purge the summary from a couple days in the past and refill it as a general policy. In general, the cost of checking the summary for accuracy is the same as the cost of refilling the summary, so just deleting and refilling is best.