Splunk Search
Highlighted

how to ignore the titile line of the csv file in the result? and display in a KV format?

Path Finder

I try to add some csv files, which contain data like the followings

Time, ACTION,ORDER_NO, ...

2009-11-2 20:00:00.041,REQUEST,48613840, ...

2009-11-2 20:00:00.041,REQUEST,48613839, ...

2009-11-2 20:00:00.041,REQUEST_ACK,48613840, ...

2009-11-2 20:00:00.041,REQUEST_ACK,48613839, ...

2009-11-2 20:00:00.046,REQUEST,48613841, ...

when I set the input source type as "csv", then the input file can be recognized with an "AutoHeader-1" stanza and a "csv-2" stanza being added to "$Splunk\etc\apps\learned\local\transforms.conf" and "$Splunk\etc\apps\learned\local\props.conf" respectively.

But I still have two problem,

  1. the first line (title line "Time, ACTION,ORDER_NO, ...") will be take as an event also, as follows

    10-4-5 02:49:28.000 _time,ACTION,ORDER_NO, ...

    2009-11-2 20:00:00.074,REQUEST,48613844, ...

    2009-11-2 20:00:00.055,REQUEST_ACK,48613842, ...

    so, how can I remove the title line from the result?

  2. How can I display the result in an KV format?

Highlighted

Re: how to ignore the titile line of the csv file in the result? and display in a KV format?

Contributor

You can add the CHECK_FOR_HEADER = true on the props.conf.

View solution in original post

Highlighted

Re: how to ignore the titile line of the csv file in the result? and display in a KV format?

Path Finder

thanks, it works for the 2nd question.

Highlighted

Re: how to ignore the titile line of the csv file in the result? and display in a KV format?

Splunk Employee
Splunk Employee
  1. I would just leave it there and ignore it/exclude it in your searches, but if you really really want to, you can apply a regex TRANSFORM to strip it out.

  2. Probably would be better if you were more specific about what you are trying to show/find. Splunk can transform results, but it's probably not very useful to simply rewrite the raw event text in a different format. What are you trying do or show with the data itself?

Highlighted

Re: how to ignore the titile line of the csv file in the result? and display in a KV format?

Contributor

in your props.conf also add this line:

[yoursourcetype]
TRANSFORMS-NoHeader = NoHeader

on your transforms.conf add this:

[NoHeader]
REGEX = Time, ACTION,ORDER_NO, ...
DEST_KEY = queue
FORMAT = nullQueue
Highlighted

Re: how to ignore the titile line of the csv file in the result? and display in a KV format?

Communicator

This is the only solution that worked for me.

0 Karma
Highlighted

Re: how to ignore the titile line of the csv file in the result? and display in a KV format?

Contributor

Seriously folks, the solution in answer <1> should be the default behaviour. Why would you want the headers included in your indexed data???

0 Karma
Highlighted

Re: how to ignore the titile line of the csv file in the result? and display in a KV format?

Super Champion

Do you really want Splunk choosing (own its own) to just drop certain events out of your log files?

Highlighted

Re: how to ignore the titile line of the csv file in the result? and display in a KV format?

Explorer

During my research into dealing with the header of a .csv, I've found that CHECK_FOR_HEADER is a deprecated feature and is no longer a best practice for dealing with the header of a .csv file.

http://docs.splunk.com/Documentation/Splunk/5.0.4/releasenotes/Deprecatedfeatures

0 Karma