csv with headers

a212830 — Mon, 28 Sep 2020 13:40:48 GMT

Hi,

I have a csv file with headers that needs processing. I want to 1) filter out the header and 2) have the fields recognized in the indexer. I tried following the steps listed in this link: http://splunk-base.splunk.com/answers/41551/how-do-i-get-auto-field-detection-on-forwarded-csv ,but it's not working.

My csv file:

"Time","IOPS","Latency(ms)","BW (MBps)"
"2013-04-03 22:06:00","9715","3.0","353.0"
"2013-04-03 22:07:00","8308","2.0","179.0"
"2013-04-03 22:08:00","6436","3.0","244.0"
"2013-04-03 22:09:00","4894","4.0","223.0"
"2013-04-03 22:10:00","4730","4.0","246.0"

input.conf:
[monitor:///fisc/dasd/xiv/perfstats/*.csv]
index=perfstats
sourcetype=xiv:perf:arrayStats
followTail = 0

props.conf:

[xiv:perf:arrayStats]
CHECK_FOR_HEADER = true
MAX_TIMESTAMP_LOOKAHEAD = 30
NO_BINARY_CHECK = 1
TIME_FORMAT = %Y-%m-%d %H:%M:$S
SHOULD_LINEMERGE = false
TIME_PREFIX = ^"
TZ = US/Eastern

TRANSFORMS-xiv:perf:arrayStats = NoHeader
TRANSFORMS-xiv:perf:arrayStats = csv-fieldextraction

transforms.conf:
[NoHeader]
REGEX = "Time","IOPS","Latency(ms)","BW (MBps)"
DEST_KEY = queue
FORMAT = nullQueue

[csv-fieldextraction]
DELIMS=","
FIELDS="Time","IOPS","Latency","BW (MBps)"
props file:

Re: csv with headers

sbrant_splunk — Fri, 05 Apr 2013 19:28:21 GMT

Have you considered bringing this all in, as a single event and then just piping it to multikv at search time? This will utilize the header as the field name and eliminate the overhead of processing this pre-index.

Re: csv with headers

a212830 — Fri, 05 Apr 2013 19:32:14 GMT

Can't be a single event - they are different timestamps associated with performance stats.

topic csv with headers in Getting Data In

csv with headers

Re: csv with headers

Re: csv with headers