topic custom datetime.xml is not working for me in Getting Data In

custom datetime.xml is not working for me

asimagu — Wed, 27 May 2015 08:29:44 GMT

hi guys

I have 2 different kind of events inside the same file. I am aware that I need to use a custom datetime.xml in order to recognize the timestamp of each sort of event. However it is not working for me.

These are the two kinds of events and I highlight in bold what should be recognized as the timestamp:

1081|2|20150512|436959|1660|0.00|1.00|0.00|4.5000|0|20

1081|2|436968|20150512|20150512|1336|1|1|000|0.00|0.00|3.20|5959|0034|G|0|20

This is what I coded in my new datetime.xml

<datetime>

<define name="_mc1" extract="year, month, day">
    &lt;![CDATA[^[^\|]*(\|[^\|]*)\|(\d\d\d\d)(\d\d)(\d\d)(\|[^\|]*){8}$]]&gt;
</define>
<define name="_mc2" extract="year, month, day, hour, minute">
    &lt;![CDATA[^[^\|]*(\|[^\|]*){3}\|(\d\d\d\d)(\d\d)(\d\d)\|(\d\d)(\d\d)(\|[^\|]*){11}$]]&gt;
</define>
<timePatterns>
     <use name="_mc2"/>
</timePatterns>
<datePatterns>
     <use name="_mc1"/>
     <use name="_mc2"/>
</datePatterns>
</datetime>

and in my props.conf this is what I have:

[test]
SHOULD_LINEMERGE = false
MAX_EVENTS = 1
DATETIME_CONFIG = /etc/apps/testing/local/datetime.xml
TZ = Europe/Madrid
MAX_TIMESTAMP_LOOKAHEAD=100

any ideas why my code is not working??

Re: custom datetime.xml is not working for me

laserval — Wed, 27 May 2015 10:57:25 GMT

The path /etc/apps/testing/local/datetime.xml looks like an absolute path from the filesystem root. Does changing it to etc/apps/testing/local/datetime.xml help?

Re: custom datetime.xml is not working for me

asimagu — Thu, 28 May 2015 08:21:41 GMT

no, same thing. The location is not the problem

Re: custom datetime.xml is not working for me

asimagu — Thu, 28 May 2015 08:24:28 GMT

So, I happened to find the solution and the source of the problem:

When doing the regex, every parenthesis is a capture group that will go into each of the timestamp variables.

I tried using ?: at the groups that I did not need to remember/capture and voila!! it works!!

This is it:

<datetime>

<define name="_mc1" extract="year, month, day">
    <text><![CDATA[^[^\|]*(?:\|[^\|]*)\|(\d\d\d\d)(\d\d)(\d\d)(?:\|[^\|]*){8}$]]></text>
</define>
<define name="_mc2" extract="year, month, day, hour, minute">
    <text><![CDATA[^[^\|]*(?:\|[^\|]*){3}\|(\d\d\d\d)(\d\d)(\d\d)\|(\d\d)(\d\d)(?:\|[^\|]*){11}$]]></text>
</define>
<timePatterns>
     <use name="_mc2"/>
</timePatterns>
<datePatterns>
     <use name="_mc1"/>
     <use name="_mc2"/>
</datePatterns>
</datetime>