How do I never freeze data in an index?

[pcjunkie]

I have one thread of data that we'd like to keep basically forever. Over the past 8 years the log has only grown to 210MB so we definitely do not have a problem with size or space.

I Splunk'd this log into it's own index, not mixed in with any other indexes so I could keep the retention settings different. The index settings are (including the defaults):

system     assureUTF8 = false
system     blockSignSize = 0
system     blockSignatureDatabase = _blocksignature
system     bucketRebuildMemoryHint = auto
system     coldPath = volume:cold1/gud
system     coldPath.maxDataSizeMB = 150000
system     coldToFrozenDir = 
system     coldToFrozenScript = 
system     compressRawdata = true
system     defaultDatabase = main
system     enableOnlineBucketRepair = true
system     enableRealtimeSearch = true
system     frozenTimePeriodInSecs = 188697600
system     homePath = volume:hot1/gud
system     homePath.maxDataSizeMB = 50000
system     indexThreads = auto
system     maxBloomBackfillBucketAge = 30d
system     maxConcurrentOptimizes = 3
system     maxDataSize = auto
system     maxHotBuckets = 3
system     maxHotIdleSecs = 0
system     maxHotSpanSecs = 7776000
system     maxMemMB = 5
system     maxMetaEntries = 1000000
system     maxRunningProcessGroups = 20
system     maxRunningProcessGroupsLowPriority = 1
system     maxTotalDataSizeMB = 500000
system     maxWarmDBCount = 300
system     memPoolMB = auto
system     minRawFileSyncSecs = disable
system     partialServiceMetaPeriod = 0
system     quarantineFutureSecs = 2592000
system     quarantinePastSecs = 77760000
system     rawChunkSizeBytes = 131072
system     rotatePeriodInSecs = 60
system     serviceMetaPeriod = 25
system     suppressBannerList = 
system     sync = 0
system     syncMeta = true
system     thawedPath = $SPLUNK_COLDDB/gud/thaweddb
system     throttleCheckPeriod = 15

The problem is this index is deleting data older than about 120 days. The total size of the index is 3MB right now.
According to the way I read this configuration it should not freeze data until it reaches 50GB (homePath.maxDataSizeMB = 50000) or approximately 5.98 years old (frozenTimePeriodInSecs = 188697600).

So why then is it deleting data from the index so soon?

[yannK]

because a bucket roll to frozen when :

• the bucket is not hot anymore
• AND all the events in the buckets are older than the frozen time policy

This is very common if your buckets are new and small.

FYI a hot bucket roll when it reaches : maxHotSpanSecs, maxHotBuckets, maxDataSize (that depends of the system, and ca go tup to 10GB per bucket)

use the| dbinspect index=myindex to check the state of your buckets.