topic Custom datetime.xml for x12 format in Getting Data In

Custom datetime.xml for x12 format

hogan24 — Mon, 28 Sep 2020 20:16:57 GMT

Trying to get datetime.xml configured to recognize a timestamp in x12 file format with no success...

Here are the possibilities of what the data could look like within the file:
[~GS*HS*123456*ASDF*20150519*0642896109*X*005010X279~
[~GS*HS*123456*ASDF20150519*064201896109*X*005010X279~
[~GS*HS*123456*ASDF20150519*06420123*896109*X*005010X279~

<datetime>
    <define name="_x12date" extract="year, month, day">
        <text><![CDATA[(?:~GS\*(?:.*?\*){3})(19\d\d|20\d\d)(0?[1-9]|1[012])(0[1-9]|[12]\d|3[01])(?:\*)]]></text>
    </define>
    <define name="_x12time1" extract="hour, minute, second, subsecond">
        <text><![CDATA[(?:~GS\*(?:.*?\*){4})(\d{2})(\d{2})(\d{2})(\d{2})(?:\*)]]></text>
    </define>
    <define name="_x12time2" extract="hour, minute, second">
        <text><![CDATA[(?:~GS\*(?:.*?\*){4})(\d{2})(\d{2})(\d{2})(?:\*)]]></text>
    </define>
    <define name="_x12time3" extract="hour, minute">
        <text><![CDATA[(?:~GS\*(?:.*?\*){4})(\d{2})(\d{2})(?:\*)]]></text>
    </define>
    <timePatterns>
          <use name="_x12time1"/>
          <use name="_x12time2"/>
          <use name="_x12time3"/>
    </timePatterns>
    <datePatterns>
          <use name="_x12date"/>
    </datePatterns>
</datetime>

props.conf looks like this:

[x12:270]
TRUNCATE = 0
DATETIME_CONFIG = /etc/apps/x12/local/datetime.xml

Any help would be appreciated as to why the timestamp is not being picked up. Thanks.

Re: Custom datetime.xml for x12 format

hogan24 — Thu, 18 Jun 2015 21:35:06 GMT

I figured this one out...I was trying to do my prefix in the regex which I believe was causing an issue. So I removed the everything in the regex up to the actual timestamp itself and put the prefix regex stanza in the TIME_PREFIX var leaving my props.conf to look like this:

[x12:270]
TRUNCATE = 0
DATETIME_CONFIG = /etc/apps/x12/local/datetime.xml
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_PREFIX = ~GS\*(?:.*?\*){3}

Here is what datetime.xml ended up looking like:

<datetime>

<define name="_year" extract="year">
    <text><![CDATA[(20\d\d|19\d\d|[901]\d(?!\d))]]></text>
</define>

<define name="_month" extract="month">
    <text><![CDATA[(0?[1-9]|1[012])(?!:)]]></text>
</define>

<define name="_day"  extract="day">
    <text><![CDATA[(0?[1-9]|[12]\d|3[01])]]></text> 
</define>

<define name="_hour" extract="hour">
    <text><![CDATA[([01]?[1-9]|[012][0-3])(?!\d)]]></text>
</define>

<define name="_minute" extract="minute">
    <text><![CDATA[([0-6]\d)(?!\d)]]></text>
</define>

<define name="_second" extract="second">
    <text><![CDATA[([0-6]\d)(?!\d)]]></text>
</define>

<define name="_x12date1" extract="year, month, day, hour, minute, second">
    <text><![CDATA[(19\d\d|20\d\d)(0?[1-9]|1[012])(0[1-9]|[12]\d|3[01])\*(\d{2})(\d{2})(\d{2})]]></text>
</define>

<define name="_x12date2" extract="year, month, day, hour, minute">
    <text><![CDATA[(19\d\d|20\d\d)(0?[1-9]|1[012])(0[1-9]|[12]\d|3[01])\*(\d{2})(\d{2})]]></text>
</define>

<timePatterns>
      <use name="_x12date1"/>
      <use name="_x12date2"/>
</timePatterns>
<datePatterns>
      <use name="_x12date1"/>
      <use name="_x12date2"/>
</datePatterns>

</datetime>

Re: Custom datetime.xml for x12 format

timothywatson — Fri, 23 Aug 2019 23:07:11 GMT

Hogan24, I am very impressed with your efforts to parse EDI X12. This appears to be a 270. Have you been successful at parsing the full transaction? Have you worked with any other HIPAA Transactions, like the 271 or the 276/277? This solution is the only hit I got regarding EDI/X12 in all of Splunkland. Can you direct me to guidance? Pretty-Please???

Re: Custom datetime.xml for x12 format

youngc_splunk — Thu, 21 Nov 2024 08:22:24 GMT

@timothywatson @hogan24 , it looks like you both are trying to implement a solution with EDIs. We now have a solutions accelerator for EDI documents. Let me know if any of you is interested in sharing some information, I can set up a call to introduce.