Getting Data In

Multiple timestap formats and subsecond with two digits only

Paolo_Prigione
Builder

Hi everybody, is it possible to teach a custom datetime.xml that my subsecond field is only two digit long?

I have a log file with two different timestamp formats, one of which is not automatically recognized. Here's an example:

00000000000000000001289525909952;2010-11-12 02:38:32:47;P2LIV ....

00000000000000000001289525916501;2010111202383591;YT00B ....

so I had to configure a custom datetime.xml which is working fine, except I am not able to let it extract the subseconds: Splunk always interprets them as zero.
Here's my custom datetime.xml:

<datetime>
<define name="_combdatetime_consMSG1" extract="year, month, day, hour, minute, second, subsecond">
    <!-- ...  2010-11-12 02:38:36:90 ...' -->
    <text><![CDATA[(\d+)-(\d+)-(\d+)\s(\d+):(\d+):(\d+):(\d+)]]></text>
</define>
<define name="_combdatetime_consMSG2" extract="year, month, day, hour, minute, second, subsecond">
    <!-- ...  2010111202383691 ...' -->
    <text><![CDATA[(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})]]></text>
</define>
<timePatterns>
      <use name="_combdatetime_consMSG1"/>
      <use name="_combdatetime_consMSG2"/>
</timePatterns>
<datePatterns>
      <use name="_combdatetime_consMSG1"/>
      <use name="_combdatetime_consMSG2"/>
</datePatterns>
</datetime>

And this is how Splunk web displays those two timestamps

11/12/10 2:38:32.000 AM

11/12/10 2:38:35.000 AM

I've tried to manually modify the source file to put a trailing 0 after the timestamp (to bring it to a millis format) and changed the regexes accordingly, but had no luck (after a full restart, too) Anybody has an idea?

Thanks, Paolo

0 Karma

dwaddle
SplunkTrust
SplunkTrust

Hi Paolo,

You may be better off using TIME_FORMAT in props.conf rather than changing datetime.xml. Its handling of subseconds and subsecond precision is covered on another answers post:

http://answers.splunk.com/questions/1946/time-format-and-subseconds

dwaddle
SplunkTrust
SplunkTrust

D'oh! I completely missed that aspect of what you were explaining.

0 Karma

Paolo_Prigione
Builder

Thanks dwaddle, I've tried the %2N to specify only two digits (or 9, on other CICS logs), but then I would have to stick with one format only and "lose" the second.

0 Karma

Paolo_Prigione
Builder

As a sidenote: MAX_TIMESTAMP_LOOKAHEAD is set high enough for Splunk to read the whole timestamp.

0 Karma
Get Updates on the Splunk Community!

How I Instrumented a Rust Application Without Knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...