How to extend Splunk log retention to forever?

[johnrbhancock]

We use splunk for data analysing and monitoring. We have the Service Now add in to collect CMDB data. It goes back and collects all the data then only collects new info on changes.  Therefore if we have any logs at any point being set from hot/cold to cold/frozen it will remove the data points we require. The add-on is not setup to grab all the data again. This means we cannot lose any of that data otherwise the results wil be incomplete. I would like to make it so that the data never goes from hot/cold cold/frozen or have some input on how we can best make this scenario work. 

[richgalloway · Accepted solution]

There is no "forever" setting for index retention.  You can set a very long retention time (10 years or more) and a large size (make sure the disk is big enough for all that data) and Splunk will keep the data long enough (probably until something forces you to reload the CMDB data).

[johnrbhancock]

Wow thankyou for such a quick response. What is the maximum for Hot > Cold. The data size is negligible 17 mb for 3 months so no issues with disk size. Comparable to the security logs its a drop in the ocean.

[richgalloway]

The highest value for frozenTimePeriodInSecs is 4294967295 (136 years).

There are a few size limit settings.  Which ones to use depend on if you use volumes or SmartStore.  Check out maxTotalDataSizeMB, maxGlobalRawDataSizeMB, maxGlobalDataSizeMB, homePath.maxDataSizeMB, and coldPath.maxDataSizeMB, all of which have the same maximum value (4294967295).