Adding new fields in an auto-extracted CSV.

howyagoin — Tue, 15 Apr 2014 10:38:27 GMT

I have had a sourcetype working fine for months which used a | as a field delimiter in a CSV file with a header.

Yesterday, the file had a few new fields added to the end of the line (header and data lines) -- the header shows this, the fields have data, the data is indexed, but the fields are not being extracted.

I have restarted Splunk, no joy. Why might Splunk 6.0.3 ignore new fields? Do I have to create an all new sourcetype?

A bit more information. My props.conf now reads:

[sdrheader2]
INDEXED_EXTRACTIONS = PSV
MAX_TIMESTAMP_LOOKAHEAD = 23
TIME_FORMAT = %d/%m/%Y %H:%M:%S
TIME_PREFIX = ^\d+\|

And I've adjusted my inputs.conf to read:

[monitor:///home/sdr/incoming]
crcSalt = <SOURCE>
initCrcLength = 2048
disabled = false
followTail = 0
index = sdr
sourcetype = sdrheader2
whitelist = \.csv$

Yet even after restarting Splunk, deleting some files which have the new header format from Splunk (| delete) and manually re-adding them with a oneshot add with the new sourcetype, the fields aren't being processed.

I've also tried making this a CSV with "FIELD_DELIMITER=|" defined, and that's what the original sdrheader sourcetype was defined as - it works for all fields except the new ones which started appearing yesterday.

The props/input.conf field are on my indexer, nothing relevant to this sourcetype on my Search Head.

Re: Adding new fields in an auto-extracted CSV.

ogdin — Wed, 23 Apr 2014 15:36:27 GMT

Are these Service Data Records?

http://docs.oracle.com/cd/E39414_01/doc.61/e29455/sbpms_sdr.htm#CHDBCIFH

topic Re: Adding new fields in an auto-extracted CSV. in Getting Data In

Adding new fields in an auto-extracted CSV.

Re: Adding new fields in an auto-extracted CSV.