Getting Data In

Multiple time formats in one sourcetype - failed to extract timestamp

Denis_B
Observer

Hello Dear Splunkers 🙂

Recently we got reported issue when our indexers are reporting multiple issues with timestamp extraction. I found out that one log file (with dedicated sourcetype) has multiple different formats of time. Therefore we are not able determine correct configuration for sourcetype iin props.conf 

Beginning of those events look like this:
1. 2021-03-22T16:15:31.995+0800
2. [DEBUG] 2021-03-22 16:15:32.075
3. <2021-03-22T02:15:29.217 CST>

Its input from *.stdout log file therefore there is a lot of different types of events.
I would like to skip letter "T" in time format definition but sadly I did not find anything that can help us.
Anybody dealt with situation like this?
Many thanks 


Denis  
The biggest problem seems to be in letter "T".

Labels (1)
0 Karma

harsmarvania57
Ultra Champion

You may achieve this using datetime.xml, have a look at my old answer post https://community.splunk.com/t5/Getting-Data-In/How-can-we-set-TIME-FORMAT-in-props-conf-where-the-m...

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Denis_B,

You have to define the timestamp format and prefix in your sourcetype when you have:

  • events in multiple rows, so you need the timestamp to define the event break,
  • timestamps in european format (dd/mm/yyyy), because Splunk by default uses the american standard (mm/dd/yyyy),
  • there are more than one timestamp in the same row and the correct one isn't the first, so Splunk takes as timestamp the first that finds.

If you haven't one of the above conditions (as it seems by your samples), you could use the auto option in timestamp definition so Splunk adapts the parsing format to the timestamp format that it finds.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...