Using Splunk v6.2.0
The default field-extraction ( sourcetype=csv
) from a CSV logfile worked fine, but it incorrectly used the seventh field as the event's timestamp.
So I created a new sourcetype, specifying DATETIME_CONFIG = CURRENT
(in props.conf) for the event's timestamp, but now it lost all the default CSV field extractions.
How can I get it to do the default field extractions from CSV without having to specify all the field names in some .conf file?
(fieldnames are in the first line of every .csv logfile)
Stanza from my props.conf:
[ctrlup]
DATETIME_CONFIG = CURRENT
INDEXED_EXTRACTIONS = csv
KV_MODE = none
CHECK_FOR_HEADER = true
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format, but using current time as event timestamp.
disabled = false
pulldown_type = true
I found this in the Splunk Docs under Create custom fields at index time
:
Note: If you are employing heavy forwarders in front of your search peers, the props and transforms processing takes place on the forwarders, not the search peers. Therefore, you must deploy the props and transforms changes to the forwarders, not the search peers.
splunkd.log also indicates that The CHECK_FOR_HEADER setting is deprecated
.
Deployed this updated props.conf
on the forwarder as well, and everything works as expected now.
[ctrlup]
DATETIME_CONFIG = CURRENT
INDEXED_EXTRACTIONS = csv
KV_MODE = auto
HEADER_FIELD_LINE_NUMBER = 2
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format, but using current time as event timestamp.
disabled = false
pulldown_type = true
Thanks.
I found this in the Splunk Docs under Create custom fields at index time
:
Note: If you are employing heavy forwarders in front of your search peers, the props and transforms processing takes place on the forwarders, not the search peers. Therefore, you must deploy the props and transforms changes to the forwarders, not the search peers.
splunkd.log also indicates that The CHECK_FOR_HEADER setting is deprecated
.
Deployed this updated props.conf
on the forwarder as well, and everything works as expected now.
[ctrlup]
DATETIME_CONFIG = CURRENT
INDEXED_EXTRACTIONS = csv
KV_MODE = auto
HEADER_FIELD_LINE_NUMBER = 2
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format, but using current time as event timestamp.
disabled = false
pulldown_type = true
Thanks.
Can you show us some of the data?
Table: Sessions
Stress Level,ID,Session Name,User,Citrix Receiver Version,State,Connect Time,Disconnect Time,Idle Time,Logon Time,Processes,ICA Latency - Last Recorded,Client IP,Client Name,Computer,Domain DNS,Initial Program,CPU,Page Faults /sec,I/O Read Operations/sec,I/O Write Operations/sec,Memory (Private Bytes),Memory (Working Set),View Client Connection Server URL,View Client Domain,View Client Protocol,View Client Type,View Client Tunnel,User Logon Server,Group Policy Load Time,Profile Load Time,Logon Duration - Other,Desktop Load Time,Logon Duration
"4.00","0","Services","","","Disconnected","","","","","61","0","0.0.0.0","","XXXX-AAAA","YYYY.com","","0.125","119.235","56.11817","0","1.040945E+09","1.223873E+09","","","","","","","0","0","0","0","0"
"3.00","119","BBBB","ZZZZ","11.3.0.41560","Active","9/1/2015 2:39:59 PM","","00:00:00","9/1/2015 2:40:47 PM","26","47","10.0.0.1","CCCC","XXXX-DDDD","YYYY.com","My Desktop","0.25","118.1605","1.851977","1.474779","6.997975E+08","9.383158E+08","","","","","","XXXX-EEE","6","31","6","1","44"
I am forwarding the .csv files from a different host using a Splunk Forwarder, if that makes a difference.
[monitor://C:\MyLogs]
crcSalt = <SOURCE>
index = ctrlup
sourcetype = ctrlup
disabled = false