- Our retention policy is 3 years for that abc index.
- When I exported the result of that query before 1 month, I was able to see that particular data
- Today when I run exact same query, I can see some missing data.
- To give you the detail, today I am seeing approx 20K less events out of 1L events.
- The date range is exact same
On your indexing layer, run the following from the command line:
splunk btool indexes list <INDEXNAME> --debug
Replacing with the name of the index that you are seeing issues with. There are a few properties to take note of:
1) coldPath.maxDataSizeMB -- The total size in MB of the Cold path for data. If this size is exceeded, data will roll to frozen (and if there is no Cold-To-Frozen archiving strategy in place, will be deleted)
2) frozenTimePeriodInSecs -- The number of seconds before data is frozen
3) maxTotalDataSizeMB -- The maximum total size across all hot/warm/cold data locations
See if any of these are lower than you expect.
Check if data is deleted because of retention or max size in last 1 month.
index=_internal sourcetype=splunkd INDEX_NAME component=BucketMover frozenTimePeriodInSecs OR maxTotalDataSizeMB