Thaw from frozen to existing cluster or new cluster or single indexer

[mikefg]

This project is to test for a potential on-prem to cloud migration.

I need to thaw several terabytes of frozen splunk data. It has been frozen over the past several years from an indexer cluster to offline repos.

The storage array where my existing indexer cluster resides doesn't have enough disk space to bring it all back. I have a secondary storage array that I can use that has plenty of space, but I can't move my existing cluster.

I need help understanding/deciding:
Should I build new indexers on the secondary array, add them to the existing cluster and thaw data to them.
Should I build a new cluster with new indexers on the secondary array and thaw the data there. 

Maybe it's easiest to just build one new standalone indexer on the secondary array and thaw all data to this one new standalone indexer?

The data will need to be searchable/exportable, I have only one search head (no search head cluster).

[richgalloway]

It's not clear how this relates to cloud migrations.

If you sign up for Splunk Cloud's Dynamic Data Self Storage (DDSS) service, then data archived in the cloud is the same as data archived on-prem.  You must thaw the data then stand up indexers to process it.

If you sign up for Splunk Cloud's Dynamic Data Active Archive (DDAA) service, then you use the GUI to tell Splunk what data to restore for you and it becomes searchable for a limited time (30 days, IIRC).  External data cannot be added to DDAA.

Either way, there's no need to migrate currently-frozen data to the cloud.

[mikefg]

Thanks for the replies. I will clarify.

Management wants me to test thawing old data so it is searchable (near term) or can be moved to cloud possibly later this year. DDSS and DDAA will be part of the discussion a bit down the road, but for now I need to test/verify thawing from frozen. We are going to retire our on-prem infrastructure at some point.

The thawed data does not have to be to our production cluster, so a standalone splunk single server would work.

If I stand up a new single instance server, is there any licensing I need to worry about if I'm just using it to thaw frozen data?

[richgalloway]

No license is needed for a standalone server that only searches thawed data since there is no ingest.

[PickleRick]

That might be a bit more complicated than that.

The main premise that for thawing data you're not ingesting anything is of course true but.

1) If you don't have a specific license, Splunk Enterprise installs with the default trial license. It has all (ok, most) of the features but it is time-limited.

2) After the trial period ends - you end up with the free license which doesn't let you schedule searches or define roles/users.

You might try to run the zero-bytes license normally meant for forwarders.

[mikefg]

Good to know, thank you! I'll start working on this and we'll see how it goes.

[livehybrid]

Hi @mikefg 

I take it you just need to thaw the data so it can be copied to your Splunk Cloud instance? Is PS doing this work? If so they might have a preference as to where this data is or how its accessed as part of the wider migration piece (there may be other bits of info I'm unaware of) (e.g. is this an Online Smartstore migration, or a Data Copy?)

However - personally (and without knowing what I dont know!) I would go with creating an instance connected to your old storage array, you actually only need a standalone Splunk instance to thaw out data and if you are not needing to do searches against this until its moved to Splunk Cloud then you shouldnt need to scale it out too much - unless you really have a lot to thaw out. Once it is thawed it will be in a format which can be used with existing processes for migrating to Splunk Cloud.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing