We have been getting messages about high percentage of small buckets. I set logging to DEBUG on one of our indexers (Windows, Splunk 7.3.4, index cluster). Using this SPL: index=_internal sourcetype=splunkd component=HotBucketRoller "finished moving" to try and see why buckets are rolling. Looking at Interesting Fields, "caller" seems to indicate the reason for the bucket roll; however, only 2 of the 4 reasons make sense to me, and I can't find them documented anywhere. The values I return are: size_exceeded, bucket_replication_failed lru, marked The first 2 are self-evident, but what are the last 2? I'm mostly interested in LRU, as that makes up 30-40% of our buckets rolls. Any insight on this? My Google-fu has failed.
... View more
