CSV Headers appearing as events

tc641 — Tue, 29 Sep 2020 15:01:33 GMT

My events are appearing on Splunk and the fields are getting extracted however the header is appearing as an event every day that the file is updated. So this appears as an event:

"user_name","name","number","opened_at"

My data is very standard:

"user_name","name","number","opened_at"
"Bob","Sarah","X10924704","2017-07-18 19:59:18"
...
...

I'm fairly new to Splunk but I have researched this problem and haven't found a solution.

So I used Splunk Web to create a props.conf which is as follows:

[sourcetype1]
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Custom
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

On Splunk Web this did not display the headers as events.

This is on my forwarder and as I said the events are appearing and fields are getting extracted.

I have tried doing this:

[sourcetype1]
TRANSFORMS-NoHeader = NoHeader

on your transforms.conf add this:
[NoHeader]
REGEX = "user_name","name","number","opened_at"
DEST_KEY = queue
FORMAT = nullQueue

Which did not work -- I'm also unsure why it would be necessary. Splunk understands that the data is a CSV so why would it think the headers are events?!

I'll also put inputs.conf and indexes.conf below (I don't know if it is necessary):

[index1]
homePath = volume:hotData/index1/db
coldPath = volume:coldData/index1/colddb
thawedPath = $SPLUNK_DB/index1/thaweddb
homePath.maxDataSizeMB = 2560
coldPath.maxDataSizeMB = 5120

[monitor://\sever\folder\file]
disabled = false
followTail = 0
index = index1
sourcetype = sourcetype1
ignoreOlderThan = 5d

Any help would be appreciated 🙂

Re: CSV Headers appearing as events

skalliger — Thu, 20 Jul 2017 11:55:34 GMT

Hi,

if your header is always in line 1, then simply set these options (found in props.conf):
Your props.conf is way too oversized for a simple csv import. Don't specify so many options you don't know exactly.

HEADER_FIELD_LINE_NUMBER = 1
HEADER_FIELD_DELIMITER = ,
FIELD_DELIMITER = ,

Not so sure if it is
= ","
or
= ,
Just try both.

Skalli

Re: CSV Headers appearing as events

tc641 — Thu, 20 Jul 2017 12:26:21 GMT

I just copied the config when splunk web created it as it was working there. I'll try this and hopefully it works 🙂

topic Re: CSV Headers appearing as events in Getting Data In

CSV Headers appearing as events

Re: CSV Headers appearing as events

Re: CSV Headers appearing as events