Hi everyone,
What is the best way to determine how many days of data each index is retaining if you only set the retention policy by max size?
In my environment I have a limited amount of storage I can use across all indexes. I have the retention poilcy set for each index to roll to frozen once it hits a max size, so that way we don't go over our allocated storage. However, we want to know how many days each index is actually retaining data.
I know on the indexer, in the "Indexes" page, it shows the earliest event for each index. However, if the earliest event in an index is 2 years ago, that doesn't mean the index is actually retaining 2 years of data. The data could either be parsed/indexed incorrectly or it could have been a newely added archived log from a machine that sent the data to that index.
Has anyone else ran into this scenario? If so, what do you do to determine the data retention for each index?
I heard metadata is faster.
| metadata index=risk type=hosts
| rename totalCount as Count firstTime as "First Event" lastTime as "Last Event"
| foreach *Event
[ eval <
This may help or it may be the same as what you see on the Indexes page.
| tstats earliest(_time) as earliest latest(_time) as latest where index=* by index
| eval range=latest-earliest
| fieldformat earliest=strftime(earliest,"%x %X")
| fieldformat latest=strftime(latest,"%x %X")
| fieldformat range=tostring(range, "duration")