error during csv index extraction

ebaileytu — Tue, 29 Sep 2020 10:45:52 GMT

I have setup a process where a heavy forwarder is ingesting a large number of csv files and the process seems to be working, but I am seeing the following error message for every single csv file

08-26-2016 08:41:19.386 -0500 WARN CsvLineBreaker - CSV StreamId: 2416848287927153596 has empty line. - data_source="/shared/storage_performance/storage/Hitachi/xxxxx/PhyLDEV_dat/xxxxx_8_26_2016_0746_PHY_Long_LDEV_1-7_0.csv", data_host="xxxxx", data_sourcetype="hitachi_perf"

Any idea what this means? I do see a blank line at the end of each CSV file. The data is being extracted as desired and the only issue I see other than the error message is the process is not very fast.

I am using 6.3.3

inputs.conf

[batch:///shared/storage_performance/storage/Hitachi]
disabled = false
host_segment = 5
index = storage
sourcetype = hitachi_perf
move_policy = sinkhole
crcSalt =
recursive = true
whitelist = .csv$

props.conf

[hitachi_perf]
category = Custom
description = Corp Hitachi Performance Data
pulldown_type = 1
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TZ = GMT
disabled = false

Thanks in advance!

Re: error during csv index extraction

jpolcari — Tue, 30 Aug 2016 13:11:08 GMT

That error is merely telling you that it found a blank line within the CSV file. Notice it is just a WARN. It appears the file is still importing without issue. I think you can safely ignore these warnings.

Re: error during csv index extraction

ebaileytu — Tue, 30 Aug 2016 14:26:02 GMT

ok - what I was thinking but nice to get some confirmation.

Re: error during csv index extraction

ATB_Jesse — Thu, 14 Jun 2018 18:54:24 GMT

Is it possible to suppress the error per-sourcetype so as not to clutter the logs?

topic Re: error during csv index extraction in Getting Data In

error during csv index extraction

Re: error during csv index extraction

Re: error during csv index extraction

Re: error during csv index extraction