topic Re: How to configure secondary storage device as Cold Destination? and move the data from hot bucket to secondary storage(cold bucket)??? in Splunk Enterprise

How to configure secondary storage device as Cold Destination? and move the data from hot bucket to secondary storage(cold bucket)???

swati_sharma — Wed, 29 Jul 2015 04:49:11 GMT

How to configure secondary storage device as Cold Destination? and move the data from hot bucket to secondary storage(cold bucket)???

Re: How to configure secondary storage device as Cold Destination? and move the data from hot bucket to secondary storage(cold bucket)???

rsennett_splunk — Wed, 29 Jul 2015 07:00:46 GMT

You can easily move it from Hot to Warm by either restarting Splunk (all hot rolls to warm) or changing the following entry in indexes.conf for the stanza matching this index :
maxHotBucket=1

To move the data from Warm to Cold you'll need to reduce the size of the index so all buckets will roll from Warm to Cold.

So... find the indexes.conf definition
edit the coldPath
temporarily change (or add) this value maxWarmDBCount=1
Everything will roll to cold if there is data still coming in. You could probably set both of them to 0 (although I've never tried it) but that seems like a runaway train...

The point is, you're using the settings on your index to force the data to roll out of hot and warm and into cold. This is of course if you still have data flowing in.

Keep in mind, as I said at the top, when you stop Splunk, all hot buckets will roll to warm. So if there is only one warm bucket, and data still flowing in... everything ends up in cold... quickly.

Re: How to configure secondary storage device as Cold Destination? and move the data from hot bucket to secondary storage(cold bucket)???

swati_sharma — Wed, 29 Jul 2015 09:35:18 GMT

I have tried with the given settings by you i.e maxHotBucket=1 and maxTotalDataSizeMB=0, However still I am not getting in the cold bucket, The behaviour is data directly move to the frozen bucket form hot bucket.

Re: How to configure secondary storage device as Cold Destination? and move the data from hot bucket to secondary storage(cold bucket)???

rsennett_splunk — Wed, 29 Jul 2015 17:32:09 GMT

You're right. I've edited my answer... check out the indexes.conf doc.
http://docs.splunk.com/Documentation/Splunk/6.2.4/admin/Indexesconf
the size directive governed the entire index... so put that back where it was. (mea culpa)
if you reduce the number of hot buckets, and then also the number of warm buckets, your stuff will have nowhere to go but cold.

Re: How to configure secondary storage device as Cold Destination? and move the data from hot bucket to secondary storage(cold bucket)???

swati_sharma — Thu, 30 Jul 2015 03:27:58 GMT

Thank you for your guidance, Can you please tell me the parameters which I need to change to get the data in cold bucket from hot bucket???

Re: How to configure secondary storage device as Cold Destination? and move the data from hot bucket to secondary storage(cold bucket)???

rsennett_splunk — Thu, 30 Jul 2015 15:31:47 GMT

swati_sharma: I changed the answer so it reflects the correct directive.
Basically, you are forcing the data to pass thru by reducing the buckets. Assuming you have data still flowing through, setting the 'bucket size' in both hot and warm to 1 will cause the data to quickly flow to cold.

All of this is documented as mentioned above.