topic Re: Export/Reindex 500GB in Getting Data In

Export/Reindex 500GB

ckillg — Wed, 24 Aug 2016 15:04:42 GMT

I'm trying to figure out the best way to send most of a 500GB index back through the Heavy Forwarder -> Indexer Cluster chain to correct hostname and sourcetype issues.

Any suggestions?

Re: Export/Reindex 500GB

gfuente — Tue, 29 Sep 2020 10:43:22 GMT

Hello

If you have a reasonable number of index/sourcetype/host combinations you can run an indivual export by combination through the api:

Example curl command from the HF (to export the file locally to the HF):

sudo curl -k -u admin:password https://splunksh:8089/services/search/jobs/export --data-urlencode search='search index=myindex sourcetype=mysourcetype host=myhost earliest=0 latest=now' -d output_mode=raw -o myindex_myhost_mysourcetype.txt

Then, create monitor stanzas in the HF with the appropiate index, sourcetype and host values for each file. You could also create transforms to get those meta data from the file name, but that will be more complex, for doing it just once

Hope this helps

Regards

Re: Export/Reindex 500GB

ckillg — Wed, 24 Aug 2016 15:45:59 GMT

This is perfect. Thank you!

Re: Export/Reindex 500GB

gfuente — Thu, 25 Aug 2016 06:44:02 GMT

Forgot to say, that you need to ensure that the HF has the appropiate props configurations for the sourctypes you want to reindex.