topic Field Extraction do not work when using the UPLOAD method in Splunk Search

Field Extraction do not work when using the UPLOAD method

Genti — Fri, 09 Jul 2010 01:21:01 GMT

Customer's issue was actually that for csv files, when setting the CHECK_FOR_HEADER=TRUE in props.conf and when uploading the file using the one time upload button through splunkweb, no automatic field extraction would happen.

I was able to reproduce this in my environment but the issue seems to go even further. When using props.conf to extract fields (at index time, this is no longer a csv-header issue) and then uploading a file, no field extractions happen at all.

Is this the default behavior? Is there any documentation about it?
Is it a bug?

Re: Field Extraction do not work when using the UPLOAD method

Genti — Fri, 09 Jul 2010 01:23:29 GMT

Asking the dev's we understand that this is not the default behavior and that something is clearly broken in the code.
The workaround, till this gets fixed, would be not to use file uploading as a means to bring data to splunk if you care for field extractions. If you use regular monitoring stanza, both index-time field extractions as well as header-checking field extractions happen without any issues.

Cheers,
.gz

Re: Field Extraction do not work when using the UPLOAD method

Stephen_Sorkin — Fri, 20 Aug 2010 03:32:32 GMT

Another workaround here is to continue to use file uploads, but manually configure the delimiter based extraction for the source or sourcetype. It should be noted that CHECK_FOR_HEADER doesn't perform any magic beyond setting a per-sourcetype search-time field extraction rule. This is easy to achieve for a person after indexing the data. The documentation at http://www.splunk.com/base/Documentation/latest/Admin/Extractfieldsfromfileheadersatindextime shows the configuration that CHECK_FOR_HEADER makes when a new input comes in.

Re: Field Extraction do not work when using the UPLOAD method

gkanapathy — Mon, 28 Sep 2020 09:16:29 GMT

In many live environments, this is necessary anyway, as CHECK_FOR_HEADER doesn't work if files are collected by a forwarder and sent to an indexer, or if you have a distributed search head separate from your indexer or forwarder.