I currently have two indexes, frozenTimePeriodInSecs=432000, and respective frozen directories outside the Splunk directory tree. Main index's maxDataSize=auto-high-volume, "Systems" index's maxDataSize=auto (undefined in stanza, so using global setting).
With this configuration, the cold buckets are unused (as is the intention) so the buckets go from warm to frozen (to the best of my understanding). Data is being successfully frozen and I've thawed it to confirm this. However, emptying out my thawed directories, I still have some data from months ago.
I can see huge breaks where all data was frozen, but other time periods have up to 4,000 counts per day. The only possible reason I can see is that all the "persistent" entries are log files with .gz extensions, but I haven't read about any issues relating to that. Any ideas behind what is causing this?
That appears to be it, some of my buckets have earliest times as many months ago but latest times as today. I assumed that upon the introduction of a lump of data (i.e., a new monitor), the buckets would be created with respect to mod-time. So if I understand correctly, once I finalize an archival process of only keeping 90 days of data, I’ll have to wait a full bucket cycle for the changes to fully take place? Would there be any complications if I find an extraneous and noisy log file, point it to the index, and sourcetype it as “unimportant” to expedite the bucket cycle?