Getting Data In

How do I emulate a tcp stream to test data ingestion?

jason0
Path Finder

Hello,

I have a stream of  call data records in xml form coming into splunk and i would like to add some ingestion-time transformations to it.  However I have broken the input at least twice, so I need a debugging setup.

I ran a packet capture to get about three minutes worth of the stream (500 or so megabytes) and stripped out the xml data into a raw text file.  I am going to "ingest" this file into a test server.

How do I dump the contents of an index so i can re-import the same data over and over again to test my transforms?

--jason

 

 

 

Labels (3)
0 Karma
1 Solution

chaker
Contributor

Hi @jason0 

Take a look at this existing community answers. In summary, you need to clean the index, and reset the fishbucket pointer for the input you are testing.

Do this in a test environment. There is no undo for these steps.

https://community.splunk.com/t5/Getting-Data-In/btprobe-and-re-indexing-data/m-p/108265

https://community.splunk.com/t5/Deployment-Architecture/Use-btprobe-reset-to-re-index-multiple-files...

https://community.splunk.com/t5/Splunk-Search/Re-indexing-multiple-files-using-btprobe/td-p/298672

Hope this helps.

View solution in original post

chaker
Contributor

Hi @jason0 

Take a look at this existing community answers. In summary, you need to clean the index, and reset the fishbucket pointer for the input you are testing.

Do this in a test environment. There is no undo for these steps.

https://community.splunk.com/t5/Getting-Data-In/btprobe-and-re-indexing-data/m-p/108265

https://community.splunk.com/t5/Deployment-Architecture/Use-btprobe-reset-to-re-index-multiple-files...

https://community.splunk.com/t5/Splunk-Search/Re-indexing-multiple-files-using-btprobe/td-p/298672

Hope this helps.

Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...