topic Re: How to configure props.conf TIME_FORMAT to recognize 2 variations of a timestamp? in Getting Data In

How to configure props.conf TIME_FORMAT to recognize 2 variations of a timestamp?

dstuder — Fri, 18 Dec 2015 19:16:21 GMT

I have a log that has time expressed like this 20151218111015. So that would be December 18th, 2015 11:10:15. However, sometimes it doesn't have the seconds. So, the props.conf TIME_FORMAT could be either %Y%m%d%H%M%S or %Y%m%d%H%M. If I put in the first, then the events without seconds don't get the correct time ... they just get the time that the event was indexed. If I put in the second, then none of my events have seconds. Is there any way around this? I would like it to keep the seconds if they exist.

Re: How to configure props.conf TIME_FORMAT to recognize 2 variations of a timestamp?

javiergn — Fri, 18 Dec 2015 20:00:44 GMT

It's a tricky one. If you have any control over the log format try to fix that first. It should be consistent.

If that's not possible then I would go for the one without seconds in your universal fw and then capture the seconds with regex and overwrite the timestamp before indexing. I can't remember whether this is possible or not for the timestamp but it's definitely doable for other meta fields such as host or source type.

Keep in mind this is going to add an unnecessary load in your indexer or heavy forwarder but if your time is not consistent you don't have that many options I'm afraid.

Thanks,
J

Re: How to configure props.conf TIME_FORMAT to recognize 2 variations of a timestamp?

sundareshr — Fri, 18 Dec 2015 20:03:19 GMT

See if this post answers your question

http://www.function1.com/2013/01/oh-no-splunking-log-files-with-multiple-formats-no-problem

Re: How to configure props.conf TIME_FORMAT to recognize 2 variations of a timestamp?

goelli — Tue, 29 Sep 2020 16:28:16 GMT

Already answered here: https://answers.splunk.com/answering/11189/view.html

If there are multiple timestamps, you can use a custom DATETIME_CONFIG instead of specifying TIME_FORMAT and TIME_PREFIX.

Re: How to configure props.conf TIME_FORMAT to recognize 2 variations of a timestamp?

effem — Thu, 07 Jun 2018 02:20:32 GMT

Exactly what i already built. Very useful. You can also recycle some stuff of the original datetime.xml