topic Re: CSV timestamp extraction issue in Getting Data In

CSV timestamp extraction issue

sains — Sun, 09 Mar 2014 04:46:00 GMT

I am indexing a simple CSV file local on the spunk server. I am trying to extract the correct time stamp from the CSV file (every line is an event) but splunk keep using the file's modified date as the timestamp. Below is the sample line from the csv file and the regex I am trying. what am I doing wrong here?
Sample line
"03/04/2014","58.71","*",""," xxxxxxxxxxx9682"

regex I have tried so far
\d{2}/\d{2}/\d{4}
^"\d{2}/\d{2}/\d{4}"
"\d{2}/\d{2}/\d{4}"

Also tried the following time format

%m/%d/%Y

Re: CSV timestamp extraction issue

lguinn2 — Sun, 09 Mar 2014 05:35:29 GMT

I would use Splunk's data preview to help with this. But add this stanza to props.conf and you should be okay. BTW, this assumes that you set the sourcetype to "myCSV" in inputs.conf

props.conf

[myCSV]
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 15
TIME_FORMAT = %m/%d/%Y
INDEXED_EXTRACTIONS = CSV
FIELD_NAMES = tstamp, field1, field2, field3, ccnum

The last 2 lines may not apply if you are not using Splunk 6.

Re: CSV timestamp extraction issue

sains — Sun, 09 Mar 2014 06:03:17 GMT

Thanks for your reply.
I tried this using data preview and advanced mode (pros.conf) but it shows the same result. I am using splunk 6.0.2. Anything else I can try?

Re: CSV timestamp extraction issue

sains — Sun, 09 Mar 2014 21:34:43 GMT

It works if my csv file has time stamp in addition to date e.g. "03/04/2014 00:00:00".
Any way to do it with just the date stamp?

Re: CSV timestamp extraction issue

nekb1958 — Mon, 10 Mar 2014 09:55:35 GMT

short test with "your" data in data preview mode

Specify timestamp format (strptime) = %m/%d/%Y
Timestamp is always prefaced by a pattern = ^"

results to the following props.conf

TIME_FORMAT=%m/%d/%Y
TIME_PREFIX=^"

works for me.

Re: CSV timestamp extraction issue

sains — Mon, 10 Mar 2014 15:44:53 GMT

This worked!. I thought I tried this before but might have missed something. thanks for your help.