How to deal with varied time formats?

[yuanliu]

I have some data containing timestamps with varied formats, e.g., sometimes "%m/%d/%y %H:%M", sometimes use "%m/%d/%Y %H:%M:%S".  Is there a format to extract these variants in one strptime()?

Currently I use

| eval good_time = strptime(bad_ts, "%m/%d/%y %H:%M"),
 good_time = if(isnull(good_time), strptime(bad_ts, "%m/%d/%Y %H:%M:%S"), good_time)

[richgalloway · Accepted solution]

Unfortunately, strptime accepts only one format string and format strings do not have an 'OR' operator.

I like your workaround, though.

[yuanliu]

---

@richgalloway wrote:

Unfortunately, strptime accepts only one format string and format strings do not have an 'OR' operator.

This would suggest that the indexer is using a more sophisticated method, because I set one of these bad time fields as TIMESTAMP_FIELDS without specifying time format, and the indexed events get the correct time. (I am really glad it does, because sourcetype would not allow me to use two formats.)

[richgalloway]

It's more than a suggestion.  Indexers do indeed have a different method for interpreting timestamps.  It's the datetime.xml file and it's a collection of regular expressions that describe just about every date/time format you can think of - and if you think of a new one you can add it to the file.

That's index-time, however.  At search time, we have much more limited options.