Getting Data In

How do I emulate a tcp stream to test data ingestion?

jason0
Path Finder

Hello,

I have a stream of  call data records in xml form coming into splunk and i would like to add some ingestion-time transformations to it.  However I have broken the input at least twice, so I need a debugging setup.

I ran a packet capture to get about three minutes worth of the stream (500 or so megabytes) and stripped out the xml data into a raw text file.  I am going to "ingest" this file into a test server.

How do I dump the contents of an index so i can re-import the same data over and over again to test my transforms?

--jason

 

 

 

Labels (3)
0 Karma
1 Solution

chaker
Contributor

Hi @jason0 

Take a look at this existing community answers. In summary, you need to clean the index, and reset the fishbucket pointer for the input you are testing.

Do this in a test environment. There is no undo for these steps.

https://community.splunk.com/t5/Getting-Data-In/btprobe-and-re-indexing-data/m-p/108265

https://community.splunk.com/t5/Deployment-Architecture/Use-btprobe-reset-to-re-index-multiple-files...

https://community.splunk.com/t5/Splunk-Search/Re-indexing-multiple-files-using-btprobe/td-p/298672

Hope this helps.

View solution in original post

chaker
Contributor

Hi @jason0 

Take a look at this existing community answers. In summary, you need to clean the index, and reset the fishbucket pointer for the input you are testing.

Do this in a test environment. There is no undo for these steps.

https://community.splunk.com/t5/Getting-Data-In/btprobe-and-re-indexing-data/m-p/108265

https://community.splunk.com/t5/Deployment-Architecture/Use-btprobe-reset-to-re-index-multiple-files...

https://community.splunk.com/t5/Splunk-Search/Re-indexing-multiple-files-using-btprobe/td-p/298672

Hope this helps.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

How to find the worst searches in your Splunk environment and how to fix them

Everyone knows Splunk is a powerful platform for running searches and doing data analytics. Your ...

Share Your Feedback: On Admin Config Service (ACS)!

Help Us Build a Better Admin Config Service Experience (ACS)   We Want Your Feedback on Admin Config Service ...