Splunk Search

How to extend Splunk log retention to forever?

johnrbhancock
Engager

We use splunk for data analysing and monitoring. We have the Service Now add in to collect CMDB data. It goes back and collects all the data then only collects new info on changes.  Therefore if we have any logs at any point being set from hot/cold to cold/frozen it will remove the data points we require. The add-on is not setup to grab all the data again. This means we cannot lose any of that data otherwise the results wil be incomplete. I would like to make it so that the data never goes from hot/cold cold/frozen or have some input on how we can best make this scenario work. 

Labels (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

There is no "forever" setting for index retention.  You can set a very long retention time (10 years or more) and a large size (make sure the disk is big enough for all that data) and Splunk will keep the data long enough (probably until something forces you to reload the CMDB data).

---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

There is no "forever" setting for index retention.  You can set a very long retention time (10 years or more) and a large size (make sure the disk is big enough for all that data) and Splunk will keep the data long enough (probably until something forces you to reload the CMDB data).

---
If this reply helps you, Karma would be appreciated.
0 Karma

johnrbhancock
Engager

Wow thankyou for such a quick response. What is the maximum for Hot > Cold. The data size is negligible 17 mb for 3 months so no issues with disk size. Comparable to the security logs its a drop in the ocean.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The highest value for frozenTimePeriodInSecs is 4294967295 (136 years).

There are a few size limit settings.  Which ones to use depend on if you use volumes or SmartStore.  Check out maxTotalDataSizeMB, maxGlobalRawDataSizeMB, maxGlobalDataSizeMB, homePath.maxDataSizeMB, and coldPath.maxDataSizeMB, all of which have the same maximum value (4294967295).

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Transform your security operations with Splunk Enterprise Security

Hi Splunk Community, Splunk Platform has set a great foundation for your security operations. With the ...

Splunk Admins and App Developers | Earn a $35 gift card!

Splunk, in collaboration with ESG (Enterprise Strategy Group) by TechTarget, is excited to announce a ...

Enterprise Security Content Update (ESCU) | New Releases

In October, the Splunk Threat Research Team had one release of new security content via the Enterprise ...