Getting Data In

Time not parsed correct suddenly

davidjehoul
Explorer

I have lines of data that looks like this (1 line) in the file source="C:\Temp\testResultLog.csv":

RT0963-01,7/02/2013 13:33:22,19/04/2013 11:13:03,0,R_1812,0,Netscape3.0,0,0,0,172.21.0.132,172.21.0.132,ohm-web-7.9.5 (d921a - 2013-05-02 13:30:00),20130502_133229,2/05/2013 20:00:02,2/05/2013 20:00:08,6,True,DAVIDJ-3500,x86,4

For the file containing these lines I have the following in the props.config file:

[testResultLog]  
CHECK_FOR_HEADER = true  
KV_MODE = none  
MAX_TIMESTAMP_LOOKAHEAD = 20  
NO_BINARY_CHECK = 1  
SHOULD_LINEMERGE = False  
TIME_FORMAT = %d/%m/%y %H:%M:%S  
TIME_PREFIX = \d{8}_\d{6},  
pulldown_type = 1

This should extract the time as indicated in bold in the data. For times before may, Splunk extracted this correct; however from may, it parses the date in the above line as the 5th of february, whereas it should parse it as the second of may.

An example of a line that was parsed correct:

BT01-02,18/03/2008 9:26:09,19/04/2013 11:11:16,0,R_1812,0,Netscape3.0,0,0,3,172.21.0.120,172.21.0.120,ohm-web-8.0.0-SNAPSHOT (c2601 - 2013-04-28 21:01:16),20130429_110040,29/04/2013 11:00:43,29/04/2013 11:02:34,111,True,DAVIDJ-3500,x86,4

Thanks for the help!

David

Tags (1)
0 Karma
1 Solution

kristian_kolb
Ultra Champion

Hi there,

It seems like you have marked (in bold) the wrong part of the message, given the TIME_PREFIX declaration. Splunk will start looking for a timestamp immediately after the TIME_PREFIX, which in this case is dddddddd_dddddd,.

Also, your TIME_FORMAT is in part wrong. You could try to change your it to;

TIME_FORMAT = %e/%m/%Y %H:%M:%S

where %e is days 1-31, instead of %d, which is 01-31
and %Y which is YYYY, instead of %y, which is YY

See http://www.strftime.net for more info.

Hope this helps,

Kristian

View solution in original post

kristian_kolb
Ultra Champion

Hi there,

It seems like you have marked (in bold) the wrong part of the message, given the TIME_PREFIX declaration. Splunk will start looking for a timestamp immediately after the TIME_PREFIX, which in this case is dddddddd_dddddd,.

Also, your TIME_FORMAT is in part wrong. You could try to change your it to;

TIME_FORMAT = %e/%m/%Y %H:%M:%S

where %e is days 1-31, instead of %d, which is 01-31
and %Y which is YYYY, instead of %y, which is YY

See http://www.strftime.net for more info.

Hope this helps,

Kristian

davidjehoul
Explorer

Indeed, I marked the wrong part. Sorry for that! Thanks for the answer!

0 Karma

Ayn
Legend

This is probably due to that you specify that Splunk should look for the 2-digit day of the month (%d) instead of the 1-digit day (%e), which is what you'll find in your logs at the beginning of the month. In other words, Splunk would fully understand the string "30/04" when you use "%d/%m", but not "1/05" (it would have to be "01/05" to be matched by the format string). Also, you have "%y" in your format string - %y is the year without century so the format string would match "13" but not "2013".

Due to this I'm pretty sure that your own format string has never worked, so Splunk has relied on its own default ones for finding valid timestamps, and these by default look for timestamps in US format where month comes before date.

A format string that would probably work better would look like

TIME_FORMAT = %e/%m/%Y %H:%M:%S

kristian_kolb
Ultra Champion

NOT AGAIN!!! I'll still leave my posting because of the TIME_PREFIX issue.

Get Updates on the Splunk Community!

Splunk Security Content for Threat Detection & Response, Q1 Roundup

Join Principal Threat Researcher, Michael Haag, as he walks through:An introduction to the Splunk Threat ...

Splunk Life | Happy Pride Month!

Happy Pride Month, Splunk Community! 🌈 In the United States, as well as many countries around the ...

SplunkTrust | Where Are They Now - Michael Uschmann

The Background Five years ago, Splunk published several videos showcasing members of the SplunkTrust to share ...