topic Re: Multiple timestap formats and subsecond with two digits only in Getting Data In

Multiple timestap formats and subsecond with two digits only

Paolo_Prigione — Tue, 16 Nov 2010 23:42:50 GMT

Hi everybody, is it possible to teach a custom datetime.xml that my subsecond field is only two digit long?

I have a log file with two different timestamp formats, one of which is not automatically recognized. Here's an example:

00000000000000000001289525909952;2010-11-12 02:38:32:47;P2LIV ....

00000000000000000001289525916501;2010111202383591;YT00B ....

so I had to configure a custom datetime.xml which is working fine, except I am not able to let it extract the subseconds: Splunk always interprets them as zero.
Here's my custom datetime.xml:

<datetime>
<define name="_combdatetime_consMSG1" extract="year, month, day, hour, minute, second, subsecond">
    <!-- ...  2010-11-12 02:38:36:90 ...' -->
    <text><![CDATA[(\d+)-(\d+)-(\d+)\s(\d+):(\d+):(\d+):(\d+)]]></text>
</define>
<define name="_combdatetime_consMSG2" extract="year, month, day, hour, minute, second, subsecond">
    <!-- ...  2010111202383691 ...' -->
    <text><![CDATA[(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})]]></text>
</define>
<timePatterns>
      <use name="_combdatetime_consMSG1"/>
      <use name="_combdatetime_consMSG2"/>
</timePatterns>
<datePatterns>
      <use name="_combdatetime_consMSG1"/>
      <use name="_combdatetime_consMSG2"/>
</datePatterns>
</datetime>

And this is how Splunk web displays those two timestamps

11/12/10 2:38:32.000 AM

11/12/10 2:38:35.000 AM

I've tried to manually modify the source file to put a trailing 0 after the timestamp (to bring it to a millis format) and changed the regexes accordingly, but had no luck (after a full restart, too) Anybody has an idea?

Thanks, Paolo

Re: Multiple timestap formats and subsecond with two digits only

Paolo_Prigione — Mon, 28 Sep 2020 09:20:50 GMT

As a sidenote: MAX_TIMESTAMP_LOOKAHEAD is set high enough for Splunk to read the whole timestamp.

Re: Multiple timestap formats and subsecond with two digits only

dwaddle — Wed, 17 Nov 2010 10:31:17 GMT

Hi Paolo,

You may be better off using TIME_FORMAT in props.conf rather than changing datetime.xml. Its handling of subseconds and subsecond precision is covered on another answers post:

http://answers.splunk.com/questions/1946/time-format-and-subseconds

Re: Multiple timestap formats and subsecond with two digits only

Paolo_Prigione — Wed, 17 Nov 2010 17:33:19 GMT

Thanks dwaddle, I've tried the %2N to specify only two digits (or 9, on other CICS logs), but then I would have to stick with one format only and "lose" the second.

Re: Multiple timestap formats and subsecond with two digits only

dwaddle — Thu, 18 Nov 2010 00:11:16 GMT

D'oh! I completely missed that aspect of what you were explaining.