Archive

How do you migrate historic data from on-prem Splunk Enterprise to Splunk Cloud?

Influencer

Hey there,

Has anyone taken the challenge of migrating historic indexed data from on-premise Splunk Enterprise to Splunk Cloud?

Happy with doing heavy lifting, etc., just trying to plan out the process of getting moved over.

Thanks,

Matt

Explorer

Yes, this is possible (we have moved data both in and out of Splunk Cloud before) but you will need to engage professional services to do this as with Splunk Cloud you have no access to copy the data onto the indexers yourself.

In our case the data was copied to an S3 bucket which you make accessible to your Splunk Cloud environment, then pro-services will work with cloud ops to copy the data to the indexers to complete the migration.

The biggest thing you will need to plan for is how much data you have to transfer to S3, if it's a relatively small amount and you have fast upload speeds, you may be able to upload direct, if you have a lot of data and/or insufficient bandwidth to copy the data in a reasonable time, an Amazon Snowball may be a viable choice.

0 Karma

Splunk Employee
Splunk Employee

This is something that is usually done via professional services. One thing to consider is, using a temporary hybrid setup until your on-prem data just ages out. You would have an on-prem search head, searching both the cloud environment (new data) and the on-prem environment (old data). Once your on-prem data ages out, you can shut down the on-prem environment. Depending on your retention requirements, this is typically an easier solution and can be less costly.

0 Karma

SplunkTrust
SplunkTrust

yes, it can be done and have been done in the past many times. is your environment clustered? how much data are we talking about? what is the reason for the data migration? or in other words, can you avoid it?
regardless, @kmorris comment is spot on, i think you should consider it.

0 Karma

Influencer

Thanks @kmorris & @adonio, our on-premise solution is running on very very old hardware, some of which is unsupported, so management keen to get that closed down rather than running for another year.

Environment is not clustered.

0 Karma