Getting Data In

Ignoring header in the csv file

Path Finder

I want to index and search csv files in splunk.

Each file has a header at the first line:

number1,number2, number3,
1,2,3
4,5,6

I've created a custom csv sourcetype in props.conf and defined custom fields i want to use instead of the header in transforms.conf:

props.conf:

[custom-csv]
DATETIME_CONFIG = CURRENT
INDEXED_EXTRACTIONS = csv
NO_BINARY_CHECK = true
category = Structured
pulldown_type = true
SHOULD_LINEMERGE = false
disabled = false
KV_MODE = none
REPORT-custom= REPORT-custom

transforms.conf:

[REPORT-custom]
DELIMS = ","
FIELDS = number1, number2, number3

However when i run a search i can still see the original field names extracted from the header plus the new ones i have defined in transforms.conf.

Is there a way to make Splunk to ignore the header line?

0 Karma

Communicator

I think the problem is that you don't need to use INDEXED_EXTRACTIONS at all if you don't intend to have Splunk extract the data based on the field headers at the top of the CSV file. I think DELIMS is also a search-time extraction and it doesn't have any reliance on INDEXED_EXTRACTIONS. So you can use PREAMBLE_REGEX but just don't use INDEXED_EXTRACTIONS. Don't treat it like a CSV file, just treat it like a log file.

SplunkTrust
SplunkTrust

Try using the PREAMBLE_REGEX attribute in props.conf. Something like this. You may need to adjust the regex.

[custom-csv]
DATETIME_CONFIG = CURRENT
INDEXED_EXTRACTIONS = csv
PREAMBLE_REGEX = number1, number2, number3
NO_BINARY_CHECK = true
category = Structured
pulldown_type = true
SHOULD_LINEMERGE = false
disabled = false
KV_MODE = none
REPORT-custom= REPORT-custom
---
If this reply helps you, an upvote would be appreciated.

Path Finder

I have tried using PREAMBLE_REGEX but now Splunk uses the next line as a header so i get 1,2,3 as the search fields, rather than values.

0 Karma

SplunkTrust
SplunkTrust

From your examples, it appears you are specifying the same field names that are already in the CSV. If so, don't bother. Omit the transform and let Splunk process the file normally.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!