topic Data restore in Splunk Cloud Platform

Data restore

vishenps — Sun, 07 Jan 2024 17:23:32 GMT

Hi Folks,
I wanted to restore a chunk of a data (jan 2023-aug 2023) from a specific index, we do use splunk cloud and use splunk's restore services.
total size of data from jan to aug: >1700GB
our licensee : 800 GB per day
will splunk reindex those data??
should I do in chunk??
I'm aware of the limitation of 10% of total archive (I'm very new to splunk tough,So correct me.)WHAT WOULD BE WAY TO GO?

Re: Data restore

richgalloway — Mon, 08 Jan 2024 00:57:39 GMT

If you use Splunk Auto Archive (DDAA) service then it will take 10 days to restore all 1.7TB of data. Each chunk restored remains searchable for 30 days so you'll have only 20 during which the whole thing can be searched. Restored data is treated much the same as thawed data in that it is indexed and searchable, but is not subject to the index retention time. Splunk Cloud automatically removes the restored after 30 days. See https://docs.splunk.com/Documentation/SplunkCloud/9.1.2308/Admin/DataArchiver#Restore_archived_data_to_Splunk_Cloud_Platform for details.

If you use Splunk's Self Service archive (DDSS) then the data must be restored to an on-prem (or private cloud) instance much the same way you would restore frozen data in Splunk Enterprise. There are no time limits for restored DDSS data. See https://docs.splunk.com/Documentation/SplunkCloud/9.1.2308/Admin/DataSelfStorage#Restore_indexed_data_from_a_self_storage_location for more.

Re: Data restore

PickleRick — Sun, 07 Jan 2024 22:23:10 GMT

Where did you get this 22 days value? I didn't find anything about restore rate limitation. Only that 10% of the overall storage entitlement. So if the OP has 800GB ingest subscription it includes 90 days of storage by default which translates to ability to restore up to 7.2TB of data at any given point in time if I understand it correctly.

(I'm not a Cloud expert, that's what I understand from Splunk websites so if I'm wrong feel free to correct me)

Re: Data restore

richgalloway — Mon, 08 Jan 2024 00:57:15 GMT

Yeah, I messed that up. I took 10% of the license rather than of the stored data. I'll fix the post.

Re: Data restore

mattymo — Thu, 25 Jan 2024 16:29:27 GMT

"should I do in chunk"? - Yes, use the date ranges to reduce your date range and restore in multiple chunks.

No it will not "reindex it" - https://docs.splunk.com/Documentation/SplunkCloud/9.1.2308/Admin/DataArchiver#Restore_archived_data_to_Splunk_Cloud_Platform

You can use the "check size" button to make sure your span is under your entitlement. Remember Dynamic Data Active Archive (DDAA) it is 10% of your Dynamic Data Active Searchable (DDAS), NOT your daily ingest entitlement. Check "cloud monitoring console> license usage > storage summary"

Span too wide! too many buckets!:

shorten the span, now i can restore!:

reduce your chunk size to under your limit, restore that data, search it, then in the table below you can clear it and restore you next chunk.

Data quality matters here, as if your timestamps are all over the place it can be suprizing how many buckets you have to restore to bring back any give date.

it will not take multiple days to restore this. if you just shrink your window you can do it in steps.

restore > search (tip use collect command to help move what you want to another index) > clear restore > repeat