Getting Data In

What is the best way to stream data out of one Splunk instance to another?

Builder

All,

We have some highly unstructured data I'd like to export from one Splunk instance to another one for testing reasons. Basically a few gigs of a subset of the data. I remember seeing a way to replay the data and stream it via TCP to another indexer, but for the life of me I can't find the docs. Any help here?

Tags (2)

Splunk Employee
Splunk Employee

I don't know if this will meet your use case, but take a look at the Splunk app for CEF. It contains a new search command called cefout and contrary to the name implication, it can send data in any format you choose to a defined routing group.
You can find more details in the documentation for the app.

Maybe this provides a decent approach to solve your problem.

0 Karma

Ultra Champion

1.) Whilst it wont work in every situation, and depending on what you need to test, you could simply add a test search head to your production indexer - this is the simplest option.
This allows you to test new apps without impacting your production environment, but using all the same data from your prod env.

2.) If you are looking to test a separate index (or maybe testing a cluster), you can configure your production indexer to forward a copy of its events to your test cluster - but this would only apply for new events going forwards.

3.) Finally, if you want to take historic data, your probably best looking at a backup and restore.

You might want to consider 2 + 3 if your needs are complex.

0 Karma