Getting Data In

12 hour timeformat without AM/PM distinction

jdoll1
Explorer

I have some data that we're ingesting that has "bad" timestamps. There is no AM/PM distinction, and they are in 12 hour format.

I have my timeformat in my props.conf as:

TIME_FORMAT = %a %b %d %Y %I:%M:%S.%3N

Will this extract the timestamps properly? It seems to in my sample extractions but I can't see how Splunk could know the AM/PM without the fields being there.

example log entries..first one is 11:26pm, second one is 12:13am. Third example is 4:34....no idea if thats AM or PM.

How does the %I know to use AM or PM if there isn't a variable present for that?

<                                  MessageUtil.java:58        > /* Sun Jul 22 2018 11:26:18.312 */  Failed to load resource bundle-BuilderMessages

<                                  MessageUtil.java:58        > /* Mon Jul 23 2018 12:13:12.000 */  Failed to load resource bundle-BuilderMessages

<                                  MessageUtil.java:58        > /* Mon Jul 23 2018 04:34:34.853 */  Failed to load resource bundle-BuilderMessages
0 Karma

VatsalJagani
Motivator

Hi @jdoll1 ,

  • Your time in event is incomplete, if this is 12 hour format then it must have specified AM/PM. You can use %p to extract time with AM/PM as %I cannot extract AM/PM.
  • Or you should consider your time in 24 hour format. In that case you can use %H instead of %I.

Coming to your question in comment, whether it will pick AM/PM if you will not provide that information in event.
So I've tested in Splunk version 7.0.1 and I'm getting same result as you are getting. So my conclusion is when you not provide AM/PM and if hour in the time is 12 then it will consider AM, in all other cases it is considering PM.

Please consider putting complete time in event if source of the event is in your hand.
Hope this helps!!!!!

0 Karma

jdoll1
Explorer

Yeah I wish I could include the AM/PM or 24 hour time. These are logs written by a vendor app that we don't have control over. I guess my 'safest' option is to just use the index time as the timestamp and not use the timestamp written in the log.

The odd thing is that Splunk seems to be ingesting the data with the correct times using %I, even though it shouldn't without the AM/PM. I just don't trust it to continue ingesting properly.

0 Karma

VatsalJagani
Motivator

If you want to use current time instead of extracting time from event, you can write below configuration into your props.conf file.

[your-sourcetype]
DATETIME_CONFIG = CURRENT

In stanza name instead of sourcetype, you can also specify source or host. Please read more details in props.conf.

0 Karma

jdoll1
Explorer

Thanks I'll give that a try.

0 Karma

jdoll1
Explorer

I added this to a dev server yesterday morning and for now it seems to be properly indexing the timestamp as AM/PM, but the question remains, how does Splunk determine it?

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!