Getting Data In

Adding new fields in an auto-extracted CSV.

howyagoin
Contributor

I have had a sourcetype working fine for months which used a | as a field delimiter in a CSV file with a header.

Yesterday, the file had a few new fields added to the end of the line (header and data lines) -- the header shows this, the fields have data, the data is indexed, but the fields are not being extracted.

I have restarted Splunk, no joy. Why might Splunk 6.0.3 ignore new fields? Do I have to create an all new sourcetype?

A bit more information. My props.conf now reads:

[sdrheader2]
INDEXED_EXTRACTIONS = PSV
MAX_TIMESTAMP_LOOKAHEAD = 23
TIME_FORMAT = %d/%m/%Y %H:%M:%S
TIME_PREFIX = ^\d+\|

And I've adjusted my inputs.conf to read:

[monitor:///home/sdr/incoming]
crcSalt = <SOURCE>
initCrcLength = 2048
disabled = false
followTail = 0
index = sdr
sourcetype = sdrheader2
whitelist = \.csv$

Yet even after restarting Splunk, deleting some files which have the new header format from Splunk (| delete) and manually re-adding them with a oneshot add with the new sourcetype, the fields aren't being processed.

I've also tried making this a CSV with "FIELD_DELIMITER=|" defined, and that's what the original sdrheader sourcetype was defined as - it works for all fields except the new ones which started appearing yesterday.

The props/input.conf field are on my indexer, nothing relevant to this sourcetype on my Search Head.

Tags (1)
0 Karma

ogdin
Splunk Employee
Splunk Employee
0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...