Getting Data In

Do I need frozen storage?

rewritex
Contributor

My retention policy has drastically changed and we are utilizing syslog as a main retention source.
On the Splunk side, I don't have a need for frozen data so I am planning just to zero it out

frozenTimePeriodInSecs = 0

Q: Is this correct?
Q: Would I just put the above line in each index.conf stanza or is there a global spot?

My goal is to just have hot/warm searchable and no cold/frozen. Future plans may be to use Splunk as a retention source and offload frozen data to another volume/location.

Thank You,
Sean

richgalloway
SplunkTrust
SplunkTrust

Presumably, you have coldToFrozenScript or coldToFrozenDir defined to freeze your data. If you remove those attributes, data will be deleted when it ages out of cold.
To not retain cold data, define a warmToColdScript for each index that simply deletes the buckets.
Note that you must restart Splunk for these changes to take effect.

---
If this reply helps you, Karma would be appreciated.
0 Karma

jkat54
SplunkTrust
SplunkTrust

When data roles to frozen it is deleted by default.

So setting that to 0 would just roll everything to frozen immediately I suppose. Typically 0 = indefinitively but it's not documented in indexes.conf spec as such.

It's really tricky to "disable" cold and I recommend you just keep it. If you must remove it, the easiest method is probably going to be a warmToColdScript that just deletes the data.

Still it would probably be better to set frozenTimePeriodInSecs to whatever value you want to keep hot/warm.., say 90 days but in seconds (90*86400) and then let Splunk do its automagic thing in relation to buckets. Also keep the path to the cold storage on the same hot/warm disk and there shouldn't be any performance impact.

0 Karma
Get Updates on the Splunk Community!

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...