topic Re: Difficulty in Timestamp Recongnition in Getting Data In

Difficulty in Timestamp Recongnition

melonman — Fri, 03 Sep 2010 12:48:56 GMT

Hi there,

I am trying to have splunk know the right timestamp in the following event.

COR_00000001,Com1,LOC_00000001,DC1,SUB_00000001,21F,GRP_00000001,Rack1,CON_00000001,Saving,8A0000000521A81D_1,2010/09/03,3F PW System,Powe,8A0000000521A81D_1,kWh,2010/09/03 00:00:00,15,83946325

There is a .csv file, and there are a header line at the first line and the rest of the lines are similar to the event above.

The correct timestamp is "2010/09/03 00:00:00" which is in %Y/%m/%d %H:%M:%S format.

My props.conf looks like the follwing, but I can not get the right timestamp.

[source::<path>]
CHECK_FOR_HEADER=false

[<sourcetype>]
SHOULD_LINEMERGE = False
BREAK_ONLY_BEFORE_DATE = False
TIME_FORMAT = %Y/%m/%d %H:%M:%S

Could anyone help me out?

Thanks!

Re: Difficulty in Timestamp Recongnition

meno — Fri, 03 Sep 2010 13:02:04 GMT

I would first try it without TIME_FORMAT but increase:

MAX_TIMESTAMP_LOOKAHEAD = <integer>
* Specifies how far (in characters) into an event Splunk should look for a timestamp.
* Defaults to 150.

Only if the result is still bad you might continue with TIME_FORMAT.

Re: Difficulty in Timestamp Recongnition

melonman — Fri, 03 Sep 2010 13:40:58 GMT

Thanks meno! it worked 🙂

Re: Difficulty in Timestamp Recongnition

Lowell — Fri, 03 Sep 2010 20:54:08 GMT

I would also recommend that you add sourcetype = <sourcetype> in your [source::<path>] stanza. Otherwise you risk the wrong sourcetype association and then your TIME_FORMAT and other sourcetype-based settings will not be applied. Splunk may be getting this right on it's own, but I've found it helpful to be explicit about sourcetype associations. That's my 2 cents.