Getting Data In

How to copy data from an index to another machine

_pravin
Contributor

Hi
I have a use case that involves copying historical data from a 3-indexer cluster (6 months old) to another machine. I have considered two potential solutions:

  1. Stop one of the indexers and copy the index from the source to the destination. The only drawback could be that the data might be incomplete, but this is not a concern as the data is for testing purposes. Given the volume of data, this process could take a significant amount of time.
  2. Create a new index using the collect command with the required set of data, then copy the index to the other machine. I believe this would be the best way to implement this use case. However, the downside is that the data volume is quite large and might take a considerable amount of time to execute the SPL in the search head, potentially affecting the performance of the SH. I am unsure if the collect command can handle such a large search and create an index. Is there a limitation on the size of the data when using the collect command?


Please advise on how to best handle this scenario.

Regards,
Pravin

Labels (2)
0 Karma
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...