topic Multiple time formats in one sourcetype - failed to extract timestamp in Getting Data In

Multiple time formats in one sourcetype - failed to extract timestamp

Denis_B — Mon, 22 Mar 2021 10:13:14 GMT

Hello Dear Splunkers 🙂

Recently we got reported issue when our indexers are reporting multiple issues with timestamp extraction. I found out that one log file (with dedicated sourcetype) has multiple different formats of time. Therefore we are not able determine correct configuration for sourcetype iin props.conf

Beginning of those events look like this:
1. 2021-03-22T16:15:31.995+0800
2. [DEBUG] 2021-03-22 16:15:32.075
3. <2021-03-22T02:15:29.217 CST>

Its input from *.stdout log file therefore there is a lot of different types of events.
I would like to skip letter "T" in time format definition but sadly I did not find anything that can help us.
Anybody dealt with situation like this?
Many thanks

Denis
The biggest problem seems to be in letter "T".

Re: Multiple time formats in one sourcetype - failed to extract timestamp

gcusello — Mon, 22 Mar 2021 10:30:12 GMT

Hi @Denis_B,

You have to define the timestamp format and prefix in your sourcetype when you have:

events in multiple rows, so you need the timestamp to define the event break,
timestamps in european format (dd/mm/yyyy), because Splunk by default uses the american standard (mm/dd/yyyy),
there are more than one timestamp in the same row and the correct one isn't the first, so Splunk takes as timestamp the first that finds.

If you haven't one of the above conditions (as it seems by your samples), you could use the auto option in timestamp definition so Splunk adapts the parsing format to the timestamp format that it finds.

Ciao.

Giuseppe

Re: Multiple time formats in one sourcetype - failed to extract timestamp

harsmarvania57 — Mon, 22 Mar 2021 12:35:52 GMT

You may achieve this using datetime.xml, have a look at my old answer post https://community.splunk.com/t5/Getting-Data-In/How-can-we-set-TIME-FORMAT-in-props-conf-where-the-milliseconds/m-p/424929