How to edit props.conf to exclude headers in CSV files from getting indexed?

[SirHill17]

Hi,

I have a CSV file with header that is monitored by Splunk. Rows are correctly read, but the headers are also included as an event row. I just want to have the header extracted as the field names (which already works at the same time).

I tried several ideas using props.conf without any success. I also had a look to the similar questions already asked by other users.

My last props.conf looks like:

[mysourcetype]
INDEXED_EXTRACTIONS = csv
HEADER_FIELD_LINE_NUMBER = 1
HEADER_FIELD_DELIMITER = ","
FIELD_DELIMITER = ","
FIELD_HEADER_REGEX = hostname,SCSI logical unit,DeviceID,SCSIBus,SCSIPort,SCSITargetId

I hope someone can help sort this out.

Thanks,

SirHill

[DUThibault]

I have had no success with the PREAMBLE_REGEX and HEADER_FIELD_LINE_NUMBER clauses... But this solution works: https://answers.splunk.com/answers/206718/how-to-pull-out-a-header-before-indexing.html ---It's a workaround, sadly, but until PREAMBLE_REGEX and HEADER_FIELD_LINE_NUMBER are fixed, that's all we have.

[woodcock · Accepted solution]

YOU MUST DEPLOY THIS ON YOUR FORWARDER. That is the problem.

[gcusello]

Did you tried to insert props.conf in your Forwarder?
Bye.
Giuseppe

[woodcock]

Did you deploy this file to your FORWARDER (not your indexers) and did you restart splunkd there?

[gcusello]

Hi SirHill17,
to exclude header from indexing you have to insert in your props.conf the following line

PREAMBLE_REGEX = <regex>

This attribute specifies a regular expression which allows Splunk to ignore these preamble lines, based on the pattern specified.
for other information see https://docs.splunk.com/Documentation/Splunk/6.5.0/Admin/Propsconf

Bye.
Giuseppe

[gcusello]

Try

FIELD_HEADER_REGEX=your_regex

Bye.
Giuseppe

[SirHill17]

As per my inital question, I already tried that and everything here:
http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Extractfieldsfromfileswithstructureddata

[gcusello]

If none of the previous options correctly runs, you could filter your header in this way:
props.conf

[your_sourcetype]
TRANSFORMS-set-remove_headers=set_OK,set_nullqueue

transforms.conf

[set_nullqueue]
REGEX=your_header_regex
DEST_KEY=queue
FORMAT=nullQueue

[set_OK]
REGEX=.
DEST_KEY = queue
FORMAT = indexQueue

Bye.
Giuseppe

[SirHill17]

Just tried adding that to my current props.conf but now it index the entire csv as one event (including the header).

props.conf is defined at the indexers level (master-node), not at the forwarder level but I don't think it changes anything.