I'm currently looking if it possible to reduce the amount of data store in index after 6 months.
I'm collecting CPU performance metric at each second for my server, after 6 months is it possible to reduce the size by keeping only CPU performance metric on 5 minutes interval for the next 6 months ?
Once data is indexed it cannot be deleted until the whole bucket ages out. There is a
delete command, but it hides data rather than remove it.
I understand this part, but when the bucket ages out, I don't need to delete everything but keep a different log interval to reduce the amount of data kept on storage.
I still need some dashboard and report but based on 5 minutes log interval instead of 1 second ?
Consider a summary index. Sample your 6-month-old data at 5-minute intervals and write it to a summary index with its own 6-month lifespan.