I want to index and search csv files in splunk.
Each file has a header at the first line:
I've created a custom csv sourcetype in props.conf and defined custom fields i want to use instead of the header in transforms.conf:
DATETIME_CONFIG = CURRENT
INDEXED_EXTRACTIONS = csv
NO_BINARY_CHECK = true
category = Structured
pulldown_type = true
SHOULD_LINEMERGE = false
disabled = false
KV_MODE = none
DELIMS = ","
FIELDS = number1, number2, number3
However when i run a search i can still see the original field names extracted from the header plus the new ones i have defined in transforms.conf.
Is there a way to make Splunk to ignore the header line?
I think the problem is that you don't need to use INDEXED_EXTRACTIONS at all if you don't intend to have Splunk extract the data based on the field headers at the top of the CSV file. I think DELIMS is also a search-time extraction and it doesn't have any reliance on INDEXED_EXTRACTIONS. So you can use PREAMBLE_REGEX but just don't use INDEXED_EXTRACTIONS. Don't treat it like a CSV file, just treat it like a log file.
Try using the
PREAMBLE_REGEX attribute in props.conf. Something like this. You may need to adjust the regex.
[custom-csv] DATETIME_CONFIG = CURRENT INDEXED_EXTRACTIONS = csv PREAMBLE_REGEX = number1, number2, number3 NO_BINARY_CHECK = true category = Structured pulldown_type = true SHOULD_LINEMERGE = false disabled = false KV_MODE = none REPORT-custom= REPORT-custom
From your examples, it appears you are specifying the same field names that are already in the CSV. If so, don't bother. Omit the transform and let Splunk process the file normally.