Getting Data In

How do I emulate a tcp stream to test data ingestion?

jason0
Path Finder

Hello,

I have a stream of  call data records in xml form coming into splunk and i would like to add some ingestion-time transformations to it.  However I have broken the input at least twice, so I need a debugging setup.

I ran a packet capture to get about three minutes worth of the stream (500 or so megabytes) and stripped out the xml data into a raw text file.  I am going to "ingest" this file into a test server.

How do I dump the contents of an index so i can re-import the same data over and over again to test my transforms?

--jason

 

 

 

Labels (3)
0 Karma
1 Solution

chaker
Contributor

Hi @jason0 

Take a look at this existing community answers. In summary, you need to clean the index, and reset the fishbucket pointer for the input you are testing.

Do this in a test environment. There is no undo for these steps.

https://community.splunk.com/t5/Getting-Data-In/btprobe-and-re-indexing-data/m-p/108265

https://community.splunk.com/t5/Deployment-Architecture/Use-btprobe-reset-to-re-index-multiple-files...

https://community.splunk.com/t5/Splunk-Search/Re-indexing-multiple-files-using-btprobe/td-p/298672

Hope this helps.

View solution in original post

chaker
Contributor

Hi @jason0 

Take a look at this existing community answers. In summary, you need to clean the index, and reset the fishbucket pointer for the input you are testing.

Do this in a test environment. There is no undo for these steps.

https://community.splunk.com/t5/Getting-Data-In/btprobe-and-re-indexing-data/m-p/108265

https://community.splunk.com/t5/Deployment-Architecture/Use-btprobe-reset-to-re-index-multiple-files...

https://community.splunk.com/t5/Splunk-Search/Re-indexing-multiple-files-using-btprobe/td-p/298672

Hope this helps.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...