timestamp recognition for elb logs

[jblphx]

I am trying to input ELB logs that are being provided to me in the following format:

elb-instance-hostname [02/10/2012:17:59:37.430204 -0000] 23.22.69.42:42455 10.158.21.55:8081 0.000046/0.148519/0.000025 200 200 0 3 "GET https://elb-instance-hostname.com/api/v1/assets/d2d9d161-1ca9-412f-a98f-4ce4049a8ee5/conversions HTTP/1.1"

Splunk is having a hard time automatically recognizing the timestamp, and as such is storing the events in the index with the incorrect date.

How would the TIME_FORMAT need to look for these events, to properly parse the timestamps?

[emiller42]

I would try the following in your props.conf stanza for this sourcetype:

TIME_FORMAT = %m/%d/%Y:%H:%M:%S.%6 %z

You can also give it some more flags to narrow down the scope of where it looks for timestamps:

TIME_PREFIX=\[

(The first '[' seen indicates that the timestamp starts on the next character)

MAX_TIMESTAMP_LOOKAHEAD  = 32

(The max number of characters it will scan past the time_prefix to extract the timestamp. Handy for performance reasons if some events are exceptionally long. In this case, your timestamp is always 32 characters long so you can get specific here)

I also find the following helpful when building strptime: strftime.net