Getting Data In

How can I fix the timestamp parsing?

New Member

I have a log file that has entries like:

2011 May 03 14:20:25:923 GMT +2 BW.AFSAdapter-AFSAdapter blablabla

So my timestamps are at the very beginning. Splunk recognizes this, but parses them slightly wrong. For example, the timestamp above is parsed to an event timestamp '5/3/11 4:20:25.923 PM'

Two strange things:

  1. It sometimes drops the '1' from the hour
  2. Sometimes 2 hours are added (we are in timezone gmt+2)

Examples:

2011 May 03 12:09:53:310 GMT +2   parsed to >>> 5/3/11 12:09:53.310 PM
2011 May 03 10:25:16:300 GMT +2   parsed to >>> 5/3/11 12:25:16.300 PM  

I have set up my props.conf as follows:

[host::*kpnnl.local]
TZ=Europe/Amsterdam

[sourcetype::tibco_bwengine]
TIME_FORMAT=%Y %b %d %H:%M:%S:%3N %Z

Any hints?

Tags (2)
0 Karma

Motivator

Pretty odd that occasionally you get the +2 hour added, and sometimes you don't. For timezones, Splunk chooses the timezone from the raw event first, if that's not available it will work off of the props.conf setting, and if that isn't available either it will use the timezone of the indexer. From your description it seems like it doesn't always recognize the timezone offset in the timestamp of the raw event.

I would suggest to leave the timezone out of your TIME_FORMAT and just use the TZ setting for the host:

[host::*kpnnl.local]
TZ=Europe/Amsterdam

[sourcetype::tibco_bwengine]
TIME_FORMAT=%Y %b %d %H:%M:%S:%3N
0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!