Hot to warm bucket issue- Why there is a deviation of rolling of data from hot to warm?

[debjit_k]

Hi all 

If my understanding is correct then data will roll from hot to warm after 90 days. I check the time on index.conf it is mentioned 90 days.

My concern

1. But for certain index I can see only see 56 days of data not 90 days.

2. A device from a index is last reporting on 30th of April now if I go and give a time frame of all time I will get no match or no data from that device. 

Can anyone guide me why there is a deviation of rolling of data from hot to warm. 

[PickleRick]

Check your buckets' status. Use

| dbinspect index=<your_index>

search over "All Time" range.

[richgalloway]

Rolling of data from hot to warm is governed by a few factors: time, size, and count.  Also, restarting an indexer will roll all hot buckets to warm.  To know why your buckets are rolling when they are we'll have to know their indexes.conf settings, how many buckets there are, and their sizes.

Whether a bucket is hot or warm has no bearing on whether data is available to search or not.  If data is only available for 56 days instead of 90 then 34 days of data moved from hot/warm/cold to frozen.

April 30th is more than 60 days ago.  If the data is only being retained for 56 days then April data probably had to be discarded to make room for newer data.  Again, we'd need to know indexer.conf settings and the nature of incoming data.

[debjit_k]

Hi @richgalloway ,

Yes i agreed with your pounts.

Total hot bucket is 3 i.e. Default 

Maxdatasize=auto I.e. 750MB by default I guess

We never restart the indexer so restarting would not be the case.

But from some device we can see 2gb of data per day is being indexing.

And one more concern I have saw few device logs which is not being reporting for 121 days.

Why is a difference and yes we have set  everything to default settings.