Timestamps and buckets

[PickleRick]

Please confirm something or correct me.

If I understand correctly, it's the event's _time that's the basis for bucket ageing (hot->warm->cold(->frozen)), right?

I understand that it's typically designed this way for collecting event which have monotonicaly "growing" time. But what would happen if my source (regardless of the reason) generated events with a "random" timestamp? One could be from an old past (several years, maybe?), another from the future and so on. Would it mean that I'd have a chance to roll the buckets after just one or two events because I'd have sufficiently old events or sufficiently  big timespan in case of hot buckets?

[richgalloway · Accepted solution]

You are correct that _time is used to put events into buckets.

Events with timestamps outside a specified range are put into "quarantine" buckets.  A quarantine bucket is a separate hot bucket that counts toward the maxHotBuckets limit.  The time range for quarantine is set by quarantinePastSecs and quarantineFutureSecs in indexes.conf.  I imagine it's possible for a quarantine bucket with a single event in it to eventually roll to warm.

[PickleRick]

Ahhh. That's the information lacking in the training 😉

Thanks for supplementing my knowledge 🙂