Restore frozen to splunk cloud

[mikefg]

Hello all.

I need to archive several years of frozen splunk data (db, rb files). The data has already been frozen and is just sitting on archive servers.

We don't want to archive the entire splunk environment (search head, indexers, etc.), just the data.

If the data is needed in the future what is the possibility of using splunk cloud as a target to thaw the data into and search? There won't be infrastructure to restore servers to, and relying on Azure, AWS, etc. to be suitable targets for old VMs is risky. 

If this is possible, what else might be needed besides the files? indexes.conf for a list of indexes?

[livehybrid]

Hi @mikefg 

If you have frozen your data on-premise, as @richgalloway said, you wont be able to just move it straight up to Splunk Cloud.

However, this is something that Splunk Professional Services can perform with you and have the appropriate tooling and processes to migrate your frozen data to Splunk Cloud. Please reach out to your account team to discuss this requirement with them.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[mikefg]

Are you saying that PS can migrate frozen data if I only have db and rb files?
   e.g.    folder_2020 | frozen data files db, rb
              folder_2021 | frozen data files db, rb
              folder_2022 | frozen data files db, rb

Or are you saying the frozen data migration has to happen while I have my current environment still active?

[richgalloway]

PS can migrate live data to your Splunk Cloud stack.  I'm pretty sure they cannot migrate frozen data (couldn't when I was in PS).  The data will live only as long as you are subscribed to Splunk Cloud, however (plus 30-ish days).

[mikefg]

So would a better option be to archive all the splunk VMs (servers) and restore if necessary in the future?

[richgalloway]

You could do that, but I don't advise it.  When those VMs are rehydrated their Splunk license will (probably) be expired so they will not function in the same distributed fashion they do now.

A better approach would be to archive a fresh download of the current Splunk software version.  When you need the archived data, launch a few VMs (a search head and some indexers), install the software, and thaw the data.  The fresh software installation will give you 30 days of distributed functionality.

You could use a single VM on which to thaw the data and search it.  That gives you an unlimited period for searching (no license expiration after the switch to Free in 30 days), but will be much slower since it's just one indexer performing the search.

Be warned that your "time capsule" of data may become the equivalent of an 8-track tape in a few years.  The current version of Splunk may not support the OS you're using (or is available) when you need the archived data.  Future versions of Splunk may not support the index format used today (although that's unlikely).

[mikefg]

Currently frozen files are stored on a series of servers.
Does the frozen data need to stay in the same location where it was frozen to? (series of servers)
Or can the files be moved to a file share or storage repo?

Is this approach valid?
Move all the frozen files from the servers they are on to a storage repo, download/store a current/same Splunk version, and current/same version of the OS. Then, if necessary in the future, build new VMs using the archived OS, install archived version of Splunk, thaw data from the repo.

[richgalloway · Accepted solution]

It's common practice to move frozen data to a different location for archiving.  This has the additional benefit of releasing space on the original server for storing new data.

Yes, that approach is valid.  Be sure to test it before you need it.

[mikefg]

Thank you for your help! Will proceed with this approach and see how it goes.

[richgalloway]

Frozen data cannot be restored to Splunk Cloud.  You must stand up your own Splunk instances (local or virtual) for thawed data.