Getting Data In

Do I need frozen storage?

rewritex
Contributor

My retention policy has drastically changed and we are utilizing syslog as a main retention source.
On the Splunk side, I don't have a need for frozen data so I am planning just to zero it out

frozenTimePeriodInSecs = 0

Q: Is this correct?
Q: Would I just put the above line in each index.conf stanza or is there a global spot?

My goal is to just have hot/warm searchable and no cold/frozen. Future plans may be to use Splunk as a retention source and offload frozen data to another volume/location.

Thank You,
Sean

richgalloway
SplunkTrust
SplunkTrust

Presumably, you have coldToFrozenScript or coldToFrozenDir defined to freeze your data. If you remove those attributes, data will be deleted when it ages out of cold.
To not retain cold data, define a warmToColdScript for each index that simply deletes the buckets.
Note that you must restart Splunk for these changes to take effect.

---
If this reply helps you, Karma would be appreciated.
0 Karma

jkat54
SplunkTrust
SplunkTrust

When data roles to frozen it is deleted by default.

So setting that to 0 would just roll everything to frozen immediately I suppose. Typically 0 = indefinitively but it's not documented in indexes.conf spec as such.

It's really tricky to "disable" cold and I recommend you just keep it. If you must remove it, the easiest method is probably going to be a warmToColdScript that just deletes the data.

Still it would probably be better to set frozenTimePeriodInSecs to whatever value you want to keep hot/warm.., say 90 days but in seconds (90*86400) and then let Splunk do its automagic thing in relation to buckets. Also keep the path to the cold storage on the same hot/warm disk and there shouldn't be any performance impact.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Persistent Queue at TcpOut — One of Splunk's Most Practical Features

Splunk introduced persistent queueing at the tcpout layer as one of the most practical resilience features in ...

Skip the Awkward Silence: Have a .conf-ersation at .conf26

Picture this. You arrive at .conf26 already having your socializing and networking plans mapped out. No ...

Rethinking Zero Trust: From Product Purchases to Logical Control Evidence

Note: This post outlines a proposed architecture and serves as an interest check. If we secure commitments ...