Getting Data In

cvs log is not getting ingested when it has only one line (in addition to the header)

mlevsh
Builder

Hi,

We came across strange issue:
cvs logs are not getting ingested when it only has only one line (in addition to the header) in a log.
The same logs with two and more lines are ingested successfully

Here are inputs.conf and  props.conf we are using

Inputs.conf
[monitor:///apps/ab_cd/resources/abcd/reports_rr/reports/abc/.../*_splunk.csv]  
sourcetype=source_type_name
index=index_name
ignoreOlderThan = 2h
crcSalt = <SOURCE>

props.conf

[source_type_name]
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
PREAMBLE_REGEX = ^Region
TIME_PREFIX= ^(?:[^,\n]*,){1}
TIME_FORMAT = %Y-%m-%d
MAX_TIMESTAMP_LOOKAHEAD=10
MAX_DAYS_HENCE = 5


Appreciate all the ideas

0 Karma

Richfez
SplunkTrust
SplunkTrust

I was expecting your props.conf to have

INDEXED_EXTRACTIONS = CSV

You are also using a TIME_PREFIX instead of TIMESTAMP_FIELDS... 

And you have a PREAMBLE_REGEX set, which looks like it's set to the first fieldname in the header (which would remove the header line) though you don't provide FIELD_NAMES...

Putting that all together, it looks like you aren't really treating those files as CSV files.  I'm not sure what's going on, but I wonder if it would work right if you treated them as CSV.

If that doesn't help, it'd be useful to see the contents of a file that doesn't work, and one that does. 

0 Karma
Get Updates on the Splunk Community!

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...