I have have some inconsistent timestamp parsing issues that I believe are due to an incorrect TIME_FORMAT value in my props.conf file and I am hoping that someone may be able to clarify what I've done wrong here. I get timestamps ingested into my Splunk instance with a format like this:
This is the TIME_FORMAT value I have been using:
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
It is mostly consistent but sometimes appears to not get parsed. Splunk Support has only been able to suggest at the Z at the end might be the issue, and on review of the documentation I don't see a specific way to note that in the TIMEFORMAT string. Does anyone know how to structure TIMEFORMAT to properly capture this?
I am not able to alter this data in any way so I must work with this format as-is.
If the 'Z' is always present in the timestamp then just include it in the format string.
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3NZ
Thanks, Rich. I tried that and several variants of the %3N bit at the end but they don't appear to have any effect -- my Splunk instance is ignoring it and ingesting events at "now". Do you have any idea why that might be? I am using a very high value for MAXTIMESTAMPLOOKAHEAD so I believe that is not the issue.
Are there any other props.conf config settings that might be interfering with this?
Ensure that you configure it on the parsing level, either HWF if available or IDX http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings and also make sure to restart the Splunk instance you configured.
btool on the parsing instance to see if your props is applied correctly or if some other
props.conf is taking precedence over it.
The Z is specific to Zulu time, i.e. UTC, i.e. GMT. In that same time zone specifier spot, you might see CDT for Central US Daylight Savings Time.
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N%Z
Please post the
_raw timestamp from a couple of the events that did not parse, and we can analyze if there is another issue.