topic Re: Extract field names from CSV header in Splunk Search

Extract field names from CSV header

JIthesh_Kumar — Wed, 03 Jul 2024 17:57:41 GMT

Hi Team,
my CSV file contains a field like bellow (1st line in CSV) How can i create transformation for field extraction

"State","Location name","Primary Number"

its retrieving a field state and Location

Expected Fields:
State
Location name
Primary Number

Re: Extract field names from CSV header

yuanliu — Wed, 03 Jul 2024 15:29:38 GMT

You needs are probably better served by INDEXED_EXTRACTIONS=csv (index time extraction) or KV_MODE=csv (search time) in sourcetype. Using regex to handle structured data like CSV is very fragile.

Re: Extract field names from CSV header

isoutamo — Sat, 06 Jul 2024 17:58:15 GMT

this is doable, but probably it needs some way to recognize which line is header line. And position in file is not that. But as @yuanliu said it's much better to use INDEXED_EXTRACTIONS=csv and then define HEADER_FIELD_LINE_NUMBER if it didn't recognize automatically that header line.

You should put props.conf also on your UF to get this work.

Structured Data Header Extraction and configuration

r. Ismo

Re: Extract field names from CSV header

PickleRick — Sat, 06 Jul 2024 19:25:12 GMT

If your column order is known and does not change, you can define delimiter-based extractions in props.conf for your sourcetype. But then you must explicitly name the fields and their order. Otherwise the only way to handle such file is using indexed extractions (which has its own drawbacks). Remember that indexed extractions happen on the initial forwarder!

Re: Extract field names from CSV header

isoutamo — Tue, 09 Jul 2024 19:51:57 GMT

If you can define which line contains headers and which values, then you can do this with any countable columns. It’s enough to known how many columns you could maximum have.