Archive2

Hot to Warm roll policy

Splunk Employee
Splunk Employee

Would like to set policy in indexes.conf to roll hot to warm on a daily basis to facilitate incremental back-up of warm databases on a daily schedule, but it appears this is not possible?

http://www.splunk.com/base/Documentation/latest/Admin/Indexesconf - does not appear to have a parameter setting for a daily scheduled roll?

http://www.splunk.com/base/Documentation/latest/Admin/Backupindexeddata tells me "Splunk rolls a hot db to a warm db based on the policy you define. By default, the main index is set to roll a hot db whenever it reaches a certain size, or it has not had any data added to it for 86400 seconds (one day), whichever occurs first. (While it is possible to force a roll of a hot db to a warm db, this is not recommended as each forced roll will permanently decrease search performance over the data. In cases where hot data needs to be backed up, a snapshot backup is the preferred method.) "

How can I achieve the timed rollover goal?

Also I'm puzzled by the purpose of maxHotIdleSecs - what is the value of rolling if there is no data being added? Can you explain perhaps as a use case?

1 Solution

Splunk Employee
Splunk Employee

You can configure Splunk to roll the hot DB's when they reach a certain size, or you can manually set up a cron job to run the roll command at regular intervals - http://www.splunk.com/base/Documentation/4.0.9/Admin/Backupindexeddata#Rolling_manually - There's no facility within Splunk to roll buckets on a set schedule.

The idle timeout is there to ensure that data does not sit in a hot DB indefinitely if that bucket is not being written to. Sometimes you will add older data to Splunk to be indexed and not fill an entire bucket. If no more data in that timerange comes into Splunk within the timeout period, the bucket will be rolled to warm DB.

View solution in original post

Splunk Employee
Splunk Employee

You can configure Splunk to roll the hot DB's when they reach a certain size, or you can manually set up a cron job to run the roll command at regular intervals - http://www.splunk.com/base/Documentation/4.0.9/Admin/Backupindexeddata#Rolling_manually - There's no facility within Splunk to roll buckets on a set schedule.

The idle timeout is there to ensure that data does not sit in a hot DB indefinitely if that bucket is not being written to. Sometimes you will add older data to Splunk to be indexed and not fill an entire bucket. If no more data in that timerange comes into Splunk within the timeout period, the bucket will be rolled to warm DB.

View solution in original post