Getting Data In

How to thaw data from frozen back into splunk?

saurabh0912
Path Finder

Hi,

We have requirement where we were asked to retrieve 3 month old data from frozen state into splunk.
We need inputs for:-
1) How to identify the buckets inside frozen data for that particular index and that time frame?
2) Will there be impact on indexer performance while thawing the data?
3)Do I need to thaw data on every indexer in cluster?
4)What would be it's impact on cluster.
5)Will it cost on license?
6)Do i have to attach disk on every indexers for thawing data?
7) How will it go back to frozen state

Thanks
Saurabh

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

1) This, perhaps, is the hardest part of thawing data. I'll assume you saved frozen buckets in a manner that preserved the index name and the bucket name ($SPLUNK_DB/<index>/db_<bucket>). If you didn't do that then I don't know how to help you.
Bucket names contain the timestamps of the oldest and newest events in them so it's just a matter of converting your time frame into epoch form and looking for buckets that timestamps in that range.

2) Yes. How much of an effect depends on how busy the indexer is and how much data is being thawed. You'll also need to restart the indexer so that's a performance impact.

3) Splunk's docs (https://docs.splunk.com/Documentation/Splunk/8.0.1/Indexer/Restorearchiveddata#Clustered_data_thawin...) recommend putting thawed buckets back on the indexer they came from, so yes.

4) The cluster will now have new buckets to track, but the effect should be negligible.

5) No

6) Only if you don't have storage for the thawed buckets

7) When you're finished with the buckets you delete them. Frozen and thawed buckets are not managed by Splunk.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

1) This, perhaps, is the hardest part of thawing data. I'll assume you saved frozen buckets in a manner that preserved the index name and the bucket name ($SPLUNK_DB/<index>/db_<bucket>). If you didn't do that then I don't know how to help you.
Bucket names contain the timestamps of the oldest and newest events in them so it's just a matter of converting your time frame into epoch form and looking for buckets that timestamps in that range.

2) Yes. How much of an effect depends on how busy the indexer is and how much data is being thawed. You'll also need to restart the indexer so that's a performance impact.

3) Splunk's docs (https://docs.splunk.com/Documentation/Splunk/8.0.1/Indexer/Restorearchiveddata#Clustered_data_thawin...) recommend putting thawed buckets back on the indexer they came from, so yes.

4) The cluster will now have new buckets to track, but the effect should be negligible.

5) No

6) Only if you don't have storage for the thawed buckets

7) When you're finished with the buckets you delete them. Frozen and thawed buckets are not managed by Splunk.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...