We, up to now, have never frozen data. However, we have a requirement now to freeze some data for years.
I need to show in a development environment how this works.
I have created a new index. Defined coldToFrozenDir and set frozenTimePeriodInSecs to 600 (10 mins).
I have created input for a text file and filled it with about 100k lines of data.
The data is being successfully indexed
The directory was created, but there is no frozen data.
I suspect it's because the data is still hot.
Is there a way to force data through the bucket cycle so I can see it show up frozen?
tried your settings on my laptop, and wrote a scheduled search that runs every 5 minutes and does that:
index = _internal | head 1000 | collect index=timtest"
try and run this search to see if its working:
index=_internal sourcetype=splunkd component=BucketMover freeze
works fine on my end
see screenshots:
did you try restarting splunk? i think restarting splunk will force the bucket to roll from hot? So you could at least test that theory and/or verify if the bucket rolls to warm/cold...
That's all it took. Restart did the trick. Interesting that the first restart created the frozendb path, but it required a second for the data to actually start freezing.
i wonder if the bucket rolls when splunk is stopping and your setting took effect as splunk was starting. So that bucket had rolled off before it knew about the directory?
@tsheets13
If you found a solution, kindly mark the question as answered so other will know what worked for you, also up-vote any helpful comments
please share your indexes.conf
. according to your description, it supposed to work fine. data will freeze regardless bucket status of time or size thresholds are met
[timtest]
coldPath = $SPLUNK_DB/timtest/colddb
homePath = $SPLUNK_DB/timtest/db
maxHotSpanSecs=900
coldToFrozenDir=$SPLUNK_DB/timtest/deeperpath/frozendb
frozenTimePeriodInSecs=600
maxTotalDataSizeMB = 512000
thawedPath = $SPLUNK_DB/timtest/thaweddb