sitop storing all the top information

[ruisantos]

Is there a way to limit the amount of summary events stored by sitop. I have scheduled search running every night with a sitop limit=20 (to store only the 20 top results) but the limit option does seem to work is and I'm storing up to 60.000 events. is there a way to resolve that?

[Stephen_Sorkin · Accepted solution]

No, the summary index accelerators will store everything, since their contract is to provide accurate answers, even when combined with other time periods. If they truncated their results, then if some truncated value were dominant in another time period, its count would be incorrect.

If you know that your distribution is relatively consistent and you don't need percentage calculation, you could just use stats:

... | stats count by f | sort - count | head 20

If you do need the total count, it's a bit trickier, but still possible:

... | stats count by f | sort - count | streamstats count as serial | eval f = if(serial > 20, "OTHER", f) | stats count by f