Issue with CSV File monitoring on Universal Forwarder

[thirusama]

Splunk Version 6.3.4

We are monitoring a csv file with same name which gets overwritten/updated in every 30 minutes. The issue seems to be intermittent i.e. it picks up the file sometime & sometimes it does not. I tried changing options like "initCrcLength" with 1024, 10240 & 1048575. None of them helped.

Since 2-3 days, I am seeing that it is reading only one line, that too partial line from the file. I have set up the inputs.conf & props.conf on Forwarder (deployed thru deployment server). Here are the current settings & the error I am getting.

inputs.conf
[monitor://C:\Temp\incident*.csv]
disabled = 0
sourcetype = imdp:ITSM:incidents_new
index = imdc_w
crcSalt = SOURCE ( with less than & greate than also included)
initCrcLength = 1048575
ignoreOlderThan = 14d

alwaysOpenFile = 1

time_before_close =15

props.conf
[imdp:ITSM:incidents_new]
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true
MAX_DAYS_AGO = 1000
MAX_DAYS_HENCE = 60
TIMESTAMP_FIELDS="opened_at"

HEADER_FIELD_LINE_NUMBER=1

[source::C:\Temp\incident.csv]
CHECK_METHOD = modtime

Here is how my sample file looks like

"number","incident_state","assignment_group","caller_id","opened_at","u_incident_assigned","u_im_service_restored_date_tim","short_description","u_im_sla_breached","severity","u_im_reporter_grp","u_im_caller_city","assigned_to","u_axp_im_config_item","u_axp_im_closureci","caused_by","u_im_causefaultychg"
"INC0000000","New","AXPVO_ABCDL","abc_name","09-23-2016 09:36:37","09-23-2016 10:00:05","","description-sample","false","Sev4","LAIBMHD_group","D.F.","","Avaya Voice","Avaya Voice","",""
----------so on----

------------But splunk did not pick any of the lines-- but just picked some intermediate line & that too half of the line..

pabc516 08:04:47 Password validation for user abc failed","false","Sev5","NGIDBA_def_AM","","name Ramadoss","APDWD516","APDWD517","",""

---- I do not see any issue with timestamp in the file for any of the rows.

This is what I see in splunkd.log.

09-23-2016 10:03:13.148 -0700 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Fri Sep 23 06:13:07 2016). Context: source::C:\Temp\incident.csv|host::WGPIS850|imdp:ITSM:incidents_new|673
host = WGPIS850 source = C:\Program Files\SplunkUniversalForwarder\var\log\splunk\splunkd.log sourcetype = splunkd

09-23-2016 10:03:13.132 -0700 INFO WatchedFile - Resetting fd to re-extract header.
host = WGPIS850 source = C:\Program Files\SplunkUniversalForwarder\var\log\splunk\splunkd.log sourcetype = splunkd

09-23-2016 10:03:13.132 -0700 INFO WatchedFile - Will begin reading at offset=0 for file='C:\Temp\incident.csv'.
host = WGPIS850 source = C:\Program Files\SplunkUniversalForwarder\var\log\splunk\splunkd.log sourcetype = splunkd

09-23-2016 10:03:13.132 -0700 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='C:\Temp\incident.csv'.

[thirusama · Accepted solution]

This issue is resolved. We were using old version of SPlunk forwarder. Now we have upgraded it to Splunk 6.3.4