Getting Data In

Why is Splunk importing header fields from CSV files as events?

nikkkc
Path Finder

Hi Guys,

I do a data Input from a folder. The folder contains CSV files. Splunk imports all the data in a correct way, except one thing: Splunk imports the header fields as an event... but why?
If I do a table search, the first row is always the same as the header....

inputs.conf:

[monitor://D:\Folder]
disabled = false
host_segment = 16
index = myindex
sourcetype = csv
whitelist = \.csv$

props.conf

[source::D:\Folder\*.csv]
TIME_FORMAT=%y/%m/%d-%H:%M:%S.%3N
0 Karma
1 Solution

inventsekar
SplunkTrust
SplunkTrust

document reference -
http://docs.splunk.com/Documentation/Splunk/6.4.2/Admin/Propsconf
on this page, search for
CHECK_FOR_HEADER = [true|false]

Try this one in your props.conf also add this line:

[yoursourcetype]
TRANSFORMS-NoHeader = NoHeader

on your transforms.conf add this:
[NoHeader]
REGEX = Time, ACTION,ORDER_NO, ...
DEST_KEY = queue
FORMAT = nullQueue

thanks and best regards,
Sekar

PS - If this or any post helped you in any way, pls consider upvoting, thanks for reading !

View solution in original post

GigaGeek
New Member

This link will contain the information you are after.
http://docs.splunk.com/Documentation/Splunk/6.2.11/Data/Extractfieldsfromfileheadersatindextime

INDEXED_EXTRACTIONS = CSV
FIELD_DELIMITER = ,

0 Karma

inventsekar
SplunkTrust
SplunkTrust

document reference -
http://docs.splunk.com/Documentation/Splunk/6.4.2/Admin/Propsconf
on this page, search for
CHECK_FOR_HEADER = [true|false]

Try this one in your props.conf also add this line:

[yoursourcetype]
TRANSFORMS-NoHeader = NoHeader

on your transforms.conf add this:
[NoHeader]
REGEX = Time, ACTION,ORDER_NO, ...
DEST_KEY = queue
FORMAT = nullQueue

thanks and best regards,
Sekar

PS - If this or any post helped you in any way, pls consider upvoting, thanks for reading !

nikkkc
Path Finder

Sorry for my impoliteness, but i am stick of trail and error....

0 Karma

inventsekar
SplunkTrust
SplunkTrust

i am not getting you. by "trial and error", you meant, you already tried this ah?

the above one is from this post and six guys upvoted that.
https://answers.splunk.com/answers/1041/how-to-ignore-the-titile-line-of-the-csv-file-in-the-result-...
so, i think this above info is a correct one.

thanks and best regards,
Sekar

PS - If this or any post helped you in any way, pls consider upvoting, thanks for reading !
0 Karma

nikkkc
Path Finder

CHeck_for_header is depricated, but i will try the second part... 🙂 thanks

0 Karma

inventsekar
SplunkTrust
SplunkTrust

nope. check_for_header is not deprecated.
the latest version 6.4.2 document reference -
http://docs.splunk.com/Documentation/Splunk/6.4.2/Admin/Propsconf
on this page, search for
CHECK_FOR_HEADER = [true|false]

thanks and best regards,
Sekar

PS - If this or any post helped you in any way, pls consider upvoting, thanks for reading !
0 Karma

nikkkc
Path Finder

OH Sorry,
they rode it years ago....
http://docs.splunk.com/Documentation/Splunk/5.0.4/releasenotes/Deprecatedfeatures
CHECK_FOR_HEADER props.conf attribute (for index-time field extractions): This feature is deprecated and might be removed in a future release.

my fault

0 Karma

nikkkc
Path Finder

it work´s with the second part:

props.conf
TIMESTAMP_FIELDS = ActualStartTime

transforms.conf
[NoHeader]
REGEX = JobHistoryID,JobID,TaskTypeID
DEST_KEY = queue
FORMAT = nullQueue

ppablo
Retired

Hi @nikkkc

If @inventsekar's answer solved your question, please don't forget to resolve the post by clicking "Accept" directly below the answer. Also, upvote the Answer or comments that you found helpful. Thanks!

Patrick

0 Karma
Get Updates on the Splunk Community!

Splunk Enterprise Security: Your Command Center for PCI DSS Compliance

Every security professional knows the drill. The PCI DSS audit is approaching, and suddenly everyone's asking ...

Developer Spotlight with Guilhem Marchand

From Splunk Engineer to Founder: The Journey Behind TrackMe    After spending over 12 years working full time ...

Cisco Catalyst Center Meets Splunk ITSI: From 'Payments Are Down' to Root Cause in ...

The Problem: When Networks and Services Don't Talk Payment systems fail at a retail location. Customers are ...