We have one index defined in indexes.conf with frozenTimePeriodInSecs as 365 days (31536000 seconds), but there are 3 years of data stored in index.It seems not working if we just define retention time period in frozenTimePeriodInSecs. Can someone help ?
$ view ./apps/launcher/local/indexes.conf
coldPath = $SPLUNK_DB/Indexname/colddb
homePath = $SPLUNK_DB/Indexname/db
thawedPath = $SPLUNK_DB/Indexname/thaweddb
frozenTimePeriodInSecs = 31536000
Could you run this search and see what values of startEpoch and endEpoch you get for your index.
| dbinspect index=YourIndex earliest=0 | table index *Epoch splunk_server path | convert ctime(*Epoch) | rename splunk_server as Indexer
This command will list all the data buckets you have for your index. If endEpoch values are newer than your data retention period, then those buckets will not be frozen.
Hi@somesoni2..I think the issue is with hot buckets only. All the data is in Hot and Warm buckets and nothing has been moved to Cold buckets. Can you guide me what best config i choose so that 365 days data is always searchable and old data should be delete.
What server did you login to check the configuration files? If you logged into the search head, then this will have no impact on retention as data lives on the indexers.
Is this a standalone setup?
Also, the data (bucket) is frozen only when the most recent event in the bucket is older then the retention period. Sometimes, a bucket can have data for varying/larger dates-range (e.g. a bucket has data for whole 1 year) and doesn't roll until the event with newest time is older than retention period. See this for more information on the same.
Sure. Run btool command on your indexers to see what frozenTimePeriodInSecs is effective.
Go to $SPLUNK_HOME/bin and then run this:
./splunk btool indexes list IndexName --debug
This should show you what the effective configuration is and from what location.