Splunk Enterprise

How to configure secondary storage device as Cold Destination? and move the data from hot bucket to secondary storage(cold bucket)???

swati_sharma
New Member

How to configure secondary storage device as Cold Destination? and move the data from hot bucket to secondary storage(cold bucket)???

Tags (2)
0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

You can easily move it from Hot to Warm by either restarting Splunk (all hot rolls to warm) or changing the following entry in indexes.conf for the stanza matching this index :
maxHotBucket=1

To move the data from Warm to Cold you'll need to reduce the size of the index so all buckets will roll from Warm to Cold.

So... find the indexes.conf definition
edit the coldPath
temporarily change (or add) this value maxWarmDBCount=1
Everything will roll to cold if there is data still coming in. You could probably set both of them to 0 (although I've never tried it) but that seems like a runaway train...

The point is, you're using the settings on your index to force the data to roll out of hot and warm and into cold. This is of course if you still have data flowing in.

Keep in mind, as I said at the top, when you stop Splunk, all hot buckets will roll to warm. So if there is only one warm bucket, and data still flowing in... everything ends up in cold... quickly.

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma

swati_sharma
New Member

Thank you for your guidance, Can you please tell me the parameters which I need to change to get the data in cold bucket from hot bucket???

0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

swati_sharma: I changed the answer so it reflects the correct directive.
Basically, you are forcing the data to pass thru by reducing the buckets. Assuming you have data still flowing through, setting the 'bucket size' in both hot and warm to 1 will cause the data to quickly flow to cold.

All of this is documented as mentioned above.

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma

swati_sharma
New Member

I have tried with the given settings by you i.e maxHotBucket=1 and maxTotalDataSizeMB=0, However still I am not getting in the cold bucket, The behaviour is data directly move to the frozen bucket form hot bucket.

0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

You're right. I've edited my answer... check out the indexes.conf doc.
http://docs.splunk.com/Documentation/Splunk/6.2.4/admin/Indexesconf
the size directive governed the entire index... so put that back where it was. (mea culpa)
if you reduce the number of hot buckets, and then also the number of warm buckets, your stuff will have nowhere to go but cold.

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...