topic Re: Time not parsed correct suddenly in Getting Data In

Time not parsed correct suddenly

davidjehoul — Mon, 28 Sep 2020 13:49:28 GMT

I have lines of data that looks like this (1 line) in the file source="C:\Temp\testResultLog.csv":

RT0963-01,7/02/2013 13:33:22,19/04/2013 11:13:03,0,R_1812,0,Netscape3.0,0,0,0,172.21.0.132,172.21.0.132,ohm-web-7.9.5 (d921a - 2013-05-02 13:30:00),20130502_133229,2/05/2013 20:00:02,2/05/2013 20:00:08,6,True,DAVIDJ-3500,x86,4

For the file containing these lines I have the following in the props.config file:

[testResultLog]  
CHECK_FOR_HEADER = true  
KV_MODE = none  
MAX_TIMESTAMP_LOOKAHEAD = 20  
NO_BINARY_CHECK = 1  
SHOULD_LINEMERGE = False  
TIME_FORMAT = %d/%m/%y %H:%M:%S  
TIME_PREFIX = \d{8}_\d{6},  
pulldown_type = 1

This should extract the time as indicated in bold in the data. For times before may, Splunk extracted this correct; however from may, it parses the date in the above line as the 5th of february, whereas it should parse it as the second of may.

An example of a line that was parsed correct:

BT01-02,18/03/2008 9:26:09,19/04/2013 11:11:16,0,R_1812,0,Netscape3.0,0,0,3,172.21.0.120,172.21.0.120,ohm-web-8.0.0-SNAPSHOT (c2601 - 2013-04-28 21:01:16),20130429_110040,29/04/2013 11:00:43,29/04/2013 11:02:34,111,True,DAVIDJ-3500,x86,4

Thanks for the help!

David

Re: Time not parsed correct suddenly

Ayn — Fri, 03 May 2013 09:13:11 GMT

This is probably due to that you specify that Splunk should look for the 2-digit day of the month (%d) instead of the 1-digit day (%e), which is what you'll find in your logs at the beginning of the month. In other words, Splunk would fully understand the string "30/04" when you use "%d/%m", but not "1/05" (it would have to be "01/05" to be matched by the format string). Also, you have "%y" in your format string - %y is the year without century so the format string would match "13" but not "2013".

Due to this I'm pretty sure that your own format string has never worked, so Splunk has relied on its own default ones for finding valid timestamps, and these by default look for timestamps in US format where month comes before date.

A format string that would probably work better would look like

TIME_FORMAT = %e/%m/%Y %H:%M:%S

Re: Time not parsed correct suddenly

kristian_kolb — Mon, 28 Sep 2020 13:49:30 GMT

Hi there,

It seems like you have marked (in bold) the wrong part of the message, given the TIME_PREFIX declaration. Splunk will start looking for a timestamp immediately after the TIME_PREFIX, which in this case is dddddddd_dddddd,.

Also, your TIME_FORMAT is in part wrong. You could try to change your it to;

TIME_FORMAT = %e/%m/%Y %H:%M:%S

where %e is days 1-31, instead of %d, which is 01-31
and %Y which is YYYY, instead of %y, which is YY

See http://www.strftime.net for more info.

Hope this helps,

Kristian

Re: Time not parsed correct suddenly

kristian_kolb — Fri, 03 May 2013 09:22:19 GMT

NOT AGAIN!!! I'll still leave my posting because of the TIME_PREFIX issue.

Re: Time not parsed correct suddenly

davidjehoul — Fri, 03 May 2013 09:48:47 GMT

Indeed, I marked the wrong part. Sorry for that! Thanks for the answer!