What is the best timestamp format to use for my custom log to be indexed by Splunk?

[ftk]

What is the best timestamp format to use for my custom log to be indexed by Splunk?

Sensible choices are:

Round-trip pattern  2010-08-06T16:43:04.1291862-04:00
Full pattern        Friday, August 06, 2010 4:47:02 PM
RFC1123 pattern     Fri, 06 Aug 2010 16:45:17 GMT
ISO 8601 pattern    2010-08-06T16:45:47
UTC sortable        2010-08-06 16:46:36Z

[vliggio]

I know this question is ancient, but it can stand using an update.

Splunk is a time based indexing system, so having correct timestamps is critical. From Splunk's logging best practices ( http://dev.splunk.com/view/logging/SP-CAAAFCK 😞

• Use the most verbose time granularity possible.
• Put the timestamp at the beginning of the line. The farther you place a timestamp from the beginning, the more difficult it is to tell it's a timestamp and not other data.
• Include a four-digit year.
• Include a time zone, preferably a GMT/UTC offset.
• Time should be rendered in microseconds in each event. The event could become detached from its original source file at some point, so having the most accurate data about an event is ideal.

I would say the "best" custom timestamp that meets all those requirements is an RFC3339 compliant timestamp (see https://tools.ietf.org/html/rfc3339 ). For example:

2017-06-27T14:59:54.158950-04:00.

The strptime (TIME_FORMAT for Splunk) would be %Y-%m-%dT%T.%6N%:z for that timestamp.

[splunkreal]

The link has a problem, copy and paste it : http://dev.splunk.com/view/logging/SP-CAAAFCK

[vliggio]

Ah, bad html link interpreter - I edited the comment to put a space before the ). Thanks!

[ftk]

Also, I am using Reound-trip pattern mostly now, Splunk parses it easily and it is easy to do in PowerShell (get-date -f s).

[ftk]

Couldn't tell ya buddy, couldn't tell ya 🙂

[Lowell]

Great question. +1 Why has no one else voted this up?

[gkanapathy · Accepted solution]

I tend to prefer the Round-trip, ISO, or UTC sortable pattern or ISO pattern with the TZ info. It doesn't matter too much if you specify an explicit format in Splunk, but it's probably best to indicate a 4-digit year, 2-digit 24-hour hours, numeric (rather than locale-specific textually named) months, and an absolutely unambiguous time zone (e.g., EST is not a good TZ). Day of the week is superfluous. Using GMT/UTC/Zulu time helps to avoid errors and problems around DST switches as well, since that zone never switches.