we are currently having some issues with an index. Basically we have configured the following in the related index:
coldPath = $SPLUNK_DB/juniper_nsm/colddb
coldToFrozenDir = $SPLUNK_DB/juniper_nsm/frozendb
homePath = $SPLUNK_DB/juniper_nsm/db
maxDataSize = auto_high_volume
maxTotalDataSizeMB = 600000
thawedPath = $SPLUNK_DB/juniper_nsm/thaweddb
frozenTimePeriodInSecs = 15552000
Theoretically with that frozen time period we should have the logs for at least 6 months online and once frozen moved to the frozendb as per configuration. The problem is that the logs get frozen before that period, its even less that 3 months. we can find the buckets to the frozendb. The index is not full and there is no other configuration for this index in the system.
One thing to mention is that this system generates a huge amount of logs nearly 20-25 gb per day.
What can be the issue that the logs get frozen before the frozentimeperiodsinsec ??Anything related to the maximum buckets in the db ? maybe if a certain nr of buckets is reached it has a larger priority than frozentimeperiod so the logs get frozen ?
Any idea ??
maxTotalDataSizeMB setting is 600000 (roughly 585 GB) . If you're logging 20 GB / day to that index, that means your buckets will roll to frozen in roughly 29 days. Buckets can be rolled to frozen if either
frozenTimePeriodInSecs is met. Check out the indexes.conf docs.
Thanks for the answer.. i will increment it . But i haven't seen this index full or nearly full .. we have an alerting policy in place that if the size is more then 90 % we get an alert. anyway i will increment this and see what happens