Getting Data In

How do I emulate a tcp stream to test data ingestion?

jason0
Path Finder

Hello,

I have a stream of  call data records in xml form coming into splunk and i would like to add some ingestion-time transformations to it.  However I have broken the input at least twice, so I need a debugging setup.

I ran a packet capture to get about three minutes worth of the stream (500 or so megabytes) and stripped out the xml data into a raw text file.  I am going to "ingest" this file into a test server.

How do I dump the contents of an index so i can re-import the same data over and over again to test my transforms?

--jason

 

 

 

Labels (3)
0 Karma
1 Solution

chaker
Contributor

Hi @jason0 

Take a look at this existing community answers. In summary, you need to clean the index, and reset the fishbucket pointer for the input you are testing.

Do this in a test environment. There is no undo for these steps.

https://community.splunk.com/t5/Getting-Data-In/btprobe-and-re-indexing-data/m-p/108265

https://community.splunk.com/t5/Deployment-Architecture/Use-btprobe-reset-to-re-index-multiple-files...

https://community.splunk.com/t5/Splunk-Search/Re-indexing-multiple-files-using-btprobe/td-p/298672

Hope this helps.

View solution in original post

chaker
Contributor

Hi @jason0 

Take a look at this existing community answers. In summary, you need to clean the index, and reset the fishbucket pointer for the input you are testing.

Do this in a test environment. There is no undo for these steps.

https://community.splunk.com/t5/Getting-Data-In/btprobe-and-re-indexing-data/m-p/108265

https://community.splunk.com/t5/Deployment-Architecture/Use-btprobe-reset-to-re-index-multiple-files...

https://community.splunk.com/t5/Splunk-Search/Re-indexing-multiple-files-using-btprobe/td-p/298672

Hope this helps.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

May 2026 Splunk Expert Sessions: Security & Observability

Level Up Your Operations: May 2026 Splunk Expert Sessions Whether you are refining your security posture or ...

Network to App: Observability Unlocked [May & June Series]

In today’s digital landscape, your environment is no longer confined to the data center. It spans complex ...

SPL2 Deep Dives, AppDynamics Integrations, SAML Made Simple and Much More on Splunk ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...