topic Re: How to recreate partial index data with metadata on different Splunk installation? in Getting Data In

How to recreate partial index data with metadata on different Splunk installation?

deepdive100 — Fri, 18 Aug 2023 18:22:50 GMT

I have a Splunk container for development (Dev). I want to import a slice of data from one index of my production Splunk (Prod) to this container so I can write searches against that data exactly as it appears in Prod.

Using Export on Prod and Import on Dev is not producing my desired outcome. Doing this as a single file with a single indexing is creating logs that are indexing the container hostname as the host not the host of the data itself. The data in the Prod index is of varying sourcetypes so the import is also only creating the sourcetype of the import file, not tha sourcetype from the data itself.

I'm looking at possibly using the EventGen app but not sure if this will do what I'm trying to do.

Is what I'm doing possible? I do not want the entire prod index. I do not want to rsync or otherwise go to the backend to move data.

EDIT: I modified the title, it seems I want the raw data and metadata to all come over in one package?

Re: How to recreate partial index data with metadata on different Splunk installation?

isoutamo — Fri, 18 Aug 2023 20:31:58 GMT

basically you could try to copy those from prod node. Here is an old post about it https://community.splunk.com/t5/Installation/How-to-migrate-indexes-to-new-indexer-instance/m-p/528064/highlight/true

You should change needed configurations after copy as you want this to be a different host also you should copy only needed indexes or remove those after rsync.

r. Ismo

Re: How to recreate partial index data with metadata on different Splunk installation?

deepdive100 — Wed, 23 Aug 2023 13:53:57 GMT

So it seems the way forward for me is to write some scripts to pull down `index=app host=each_host sourcetype=each_sourcetype` for a specific time block, export them with the hostname in the title and import each with the hostname widget set to the filename. One script of API calls with the variables on the hosts and sourcetype should do it. Will try it out and update here