Getting Data In

custom datetime.xml is not working for me

asimagu
Builder

hi guys

I have 2 different kind of events inside the same file. I am aware that I need to use a custom datetime.xml in order to recognize the timestamp of each sort of event. However it is not working for me.

These are the two kinds of events and I highlight in bold what should be recognized as the timestamp:

1081|2|20150512|436959|1660|0.00|1.00|0.00|4.5000|0|20

1081|2|436968|20150512|20150512|1336|1|1|000|0.00|0.00|3.20|5959|0034|G|0|20

This is what I coded in my new datetime.xml

<datetime>

<define name="_mc1" extract="year, month, day">
    &lt;![CDATA[^[^\|]*(\|[^\|]*)\|(\d\d\d\d)(\d\d)(\d\d)(\|[^\|]*){8}$]]&gt;
</define>
<define name="_mc2" extract="year, month, day, hour, minute">
    &lt;![CDATA[^[^\|]*(\|[^\|]*){3}\|(\d\d\d\d)(\d\d)(\d\d)\|(\d\d)(\d\d)(\|[^\|]*){11}$]]&gt;
</define>
<timePatterns>
     <use name="_mc2"/>
</timePatterns>
<datePatterns>
     <use name="_mc1"/>
     <use name="_mc2"/>
</datePatterns>
</datetime>

and in my props.conf this is what I have:

[test]
SHOULD_LINEMERGE = false
MAX_EVENTS = 1
DATETIME_CONFIG = /etc/apps/testing/local/datetime.xml
TZ = Europe/Madrid
MAX_TIMESTAMP_LOOKAHEAD=100

any ideas why my code is not working??

Tags (2)
1 Solution

asimagu
Builder

So, I happened to find the solution and the source of the problem:

When doing the regex, every parenthesis is a capture group that will go into each of the timestamp variables.

I tried using ?: at the groups that I did not need to remember/capture and voila!! it works!!

This is it:

<datetime>

<define name="_mc1" extract="year, month, day">
    <text><![CDATA[^[^\|]*(?:\|[^\|]*)\|(\d\d\d\d)(\d\d)(\d\d)(?:\|[^\|]*){8}$]]></text>
</define>
<define name="_mc2" extract="year, month, day, hour, minute">
    <text><![CDATA[^[^\|]*(?:\|[^\|]*){3}\|(\d\d\d\d)(\d\d)(\d\d)\|(\d\d)(\d\d)(?:\|[^\|]*){11}$]]></text>
</define>
<timePatterns>
     <use name="_mc2"/>
</timePatterns>
<datePatterns>
     <use name="_mc1"/>
     <use name="_mc2"/>
</datePatterns>
</datetime>

View solution in original post

asimagu
Builder

So, I happened to find the solution and the source of the problem:

When doing the regex, every parenthesis is a capture group that will go into each of the timestamp variables.

I tried using ?: at the groups that I did not need to remember/capture and voila!! it works!!

This is it:

<datetime>

<define name="_mc1" extract="year, month, day">
    <text><![CDATA[^[^\|]*(?:\|[^\|]*)\|(\d\d\d\d)(\d\d)(\d\d)(?:\|[^\|]*){8}$]]></text>
</define>
<define name="_mc2" extract="year, month, day, hour, minute">
    <text><![CDATA[^[^\|]*(?:\|[^\|]*){3}\|(\d\d\d\d)(\d\d)(\d\d)\|(\d\d)(\d\d)(?:\|[^\|]*){11}$]]></text>
</define>
<timePatterns>
     <use name="_mc2"/>
</timePatterns>
<datePatterns>
     <use name="_mc1"/>
     <use name="_mc2"/>
</datePatterns>
</datetime>

laserval
Communicator

The path /etc/apps/testing/local/datetime.xml looks like an absolute path from the filesystem root. Does changing it to etc/apps/testing/local/datetime.xml help?

0 Karma

asimagu
Builder

no, same thing. The location is not the problem

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...