Splunk Search

How to search non-header CSV format data with Hunk?

cwl
Contributor

I am using Hunk 6.2.1 and I have some csv format data saved in my hadoop cluster which doesn't have csv header.
By default it seems like Hunk is recognizing the first line of my csv format data as header and not data.
Is there a way to search this kind of data so Hunk doesn't think that the first line content is csv header ?

Tags (3)
1 Solution

cwl
Contributor

By default, Hunk will use SimpleCSVRecordReader reader to process csv files and convert csv to json.
By using SplunkLineRecordReader, you can avoid this conversion which enables you to use Splunk Enterprise's search time field extractions settings.
So you will need to use the default line reader to let the splunk parsing pipelines do all the event breaking etc.
To use the default line reader you will need to add "vix.input.1.recordreader = SplunkLineRecordReader" in you virtual index.

[sampledata] 
vix.input.1.path = /data/... 
vix.provider = cdh4 
vix.input.1.recordreader = SplunkLineRecordReader 

View solution in original post

cwl
Contributor

By default, Hunk will use SimpleCSVRecordReader reader to process csv files and convert csv to json.
By using SplunkLineRecordReader, you can avoid this conversion which enables you to use Splunk Enterprise's search time field extractions settings.
So you will need to use the default line reader to let the splunk parsing pipelines do all the event breaking etc.
To use the default line reader you will need to add "vix.input.1.recordreader = SplunkLineRecordReader" in you virtual index.

[sampledata] 
vix.input.1.path = /data/... 
vix.provider = cdh4 
vix.input.1.recordreader = SplunkLineRecordReader 

burwell
SplunkTrust
SplunkTrust

Thanks SO much for this post. I can't find SplunkLineRecordReader documented anywhere.

We had non-compressed CSV data that was comma separated. When I used SimpleCSVRecordReader it did indeed treat the data as JSON. Not sure why? And this was bad as I was getting pairs of data.

The solution was vix.input.1.recordreader = SplunkLineRecordReader as listed above.

Thanks again.

0 Karma
Get Updates on the Splunk Community!

Unleash the Power of Splunk MCP and AI, Meet Us at .Conf 2025, and Find Even More New ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Observability Professionals: Build Resilience and Visibility with These .conf25 ...

  If you're focused on performance, availability, and full-stack visibility, the Observability track at ...

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...