Avoid header at Index time and extract field during search

[emaccaferri]

Hi,

I'm trying to not have in my events the header of a txt file separated by tab (I suppose it's at index time), after that I extract field at search time. The problem is that everything I tried left the header in the event.
Any key?

Here my code in props.conf and transfomrs.conf:

props.conf

[mysourcetype]
NO_BINARY_CHECK = 1
pulldown_type = 1
TRANSFORMS-NoHeader = NoHeader
REPORT -mysourcetypeextract = extract_mysourcetype

transforms.conf

 [extract_mysourcetype]
 DELIMS = "\t"
 FIELDS = personid, email,  createdate, time_window_web

[NoHeader]
REGEX = ^"Personid".*
DEST_KEY = queue
FORMAT = nullQueue

and the first lines of my log file

"Personid"  "Email" "Createdate"    "Time Window Web"   
"0000000"   "xxxxxx@xxxx.xxx"   "00/00/1900"    "00:00"

I really don't know what to do, I've tried all the answer on the site 🙂

Thank you for the help
Erica

[ogdin]

When you use Header-based Index-time field extractions, the header doesn't get indexed but we use it to map the field:

http://docs.splunk.com/Documentation/Splunk/latest/Data/Extractfieldsfromfileheadersatindextime

Try in props.conf:


FIELD_DELIMITER=space
FIELD_QUOTE="
HEADER_FIELD_DELIMITER=space
HEADER_FIELD_QUOTE="


You will need to play with the settings a bit if tabs separate the header or events but this is way better than stripping the header as it skips the need for doing search-time field extraction.

I should note this also only works on Splunk 6.

[_d_ · Accepted solution]

This seems to work:

[NoHeader]
REGEX = ^"Personid"
DEST_KEY = queue
FORMAT = nullQueue

and also your original:

[NoHeader]
REGEX = ^"Personid".*
DEST_KEY = queue
FORMAT = nullQueue

Make sure you're doing this on an indexer (or heavy forwarder) and restart after change.

[emaccaferri]

Yes, it was working, but testing on the same file I had to cancel all the index and the data