Getting Data In

Custom datetime.xml for x12 format

Path Finder

Trying to get datetime.xml configured to recognize a timestamp in x12 file format with no success...

Here are the possibilities of what the data could look like within the file:
[~GS*HS*123456*ASDF*20150519*0642896109*X*005010X279~
[~GS*HS*123456*ASDF
20150519*064201896109*X*005010X279~
[~GS*HS*123456*ASDF
20150519*06420123*896109*X*005010X279~

<datetime>
    <define name="_x12date" extract="year, month, day">
        <text><![CDATA[(?:~GS\*(?:.*?\*){3})(19\d\d|20\d\d)(0?[1-9]|1[012])(0[1-9]|[12]\d|3[01])(?:\*)]]></text>
    </define>
    <define name="_x12time1" extract="hour, minute, second, subsecond">
        <text><![CDATA[(?:~GS\*(?:.*?\*){4})(\d{2})(\d{2})(\d{2})(\d{2})(?:\*)]]></text>
    </define>
    <define name="_x12time2" extract="hour, minute, second">
        <text><![CDATA[(?:~GS\*(?:.*?\*){4})(\d{2})(\d{2})(\d{2})(?:\*)]]></text>
    </define>
    <define name="_x12time3" extract="hour, minute">
        <text><![CDATA[(?:~GS\*(?:.*?\*){4})(\d{2})(\d{2})(?:\*)]]></text>
    </define>
    <timePatterns>
          <use name="_x12time1"/>
          <use name="_x12time2"/>
          <use name="_x12time3"/>
    </timePatterns>
    <datePatterns>
          <use name="_x12date"/>
    </datePatterns>
</datetime>

props.conf looks like this:

[x12:270]
TRUNCATE = 0
DATETIME_CONFIG = /etc/apps/x12/local/datetime.xml

Any help would be appreciated as to why the timestamp is not being picked up. Thanks.

Tags (1)
1 Solution

Path Finder

I figured this one out...I was trying to do my prefix in the regex which I believe was causing an issue. So I removed the everything in the regex up to the actual timestamp itself and put the prefix regex stanza in the TIME_PREFIX var leaving my props.conf to look like this:

[x12:270]
TRUNCATE = 0
DATETIME_CONFIG = /etc/apps/x12/local/datetime.xml
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_PREFIX = ~GS\*(?:.*?\*){3}

Here is what datetime.xml ended up looking like:

<datetime>

<define name="_year" extract="year">
    <text><![CDATA[(20\d\d|19\d\d|[901]\d(?!\d))]]></text>
</define>

<define name="_month" extract="month">
    <text><![CDATA[(0?[1-9]|1[012])(?!:)]]></text>
</define>

<define name="_day"  extract="day">
    <text><![CDATA[(0?[1-9]|[12]\d|3[01])]]></text> 
</define>

<define name="_hour" extract="hour">
    <text><![CDATA[([01]?[1-9]|[012][0-3])(?!\d)]]></text>
</define>

<define name="_minute" extract="minute">
    <text><![CDATA[([0-6]\d)(?!\d)]]></text>
</define>

<define name="_second" extract="second">
    <text><![CDATA[([0-6]\d)(?!\d)]]></text>
</define>

<define name="_x12date1" extract="year, month, day, hour, minute, second">
    <text><![CDATA[(19\d\d|20\d\d)(0?[1-9]|1[012])(0[1-9]|[12]\d|3[01])\*(\d{2})(\d{2})(\d{2})]]></text>
</define>

<define name="_x12date2" extract="year, month, day, hour, minute">
    <text><![CDATA[(19\d\d|20\d\d)(0?[1-9]|1[012])(0[1-9]|[12]\d|3[01])\*(\d{2})(\d{2})]]></text>
</define>

<timePatterns>
      <use name="_x12date1"/>
      <use name="_x12date2"/>
</timePatterns>
<datePatterns>
      <use name="_x12date1"/>
      <use name="_x12date2"/>
</datePatterns>

</datetime>

View solution in original post

Path Finder

I figured this one out...I was trying to do my prefix in the regex which I believe was causing an issue. So I removed the everything in the regex up to the actual timestamp itself and put the prefix regex stanza in the TIME_PREFIX var leaving my props.conf to look like this:

[x12:270]
TRUNCATE = 0
DATETIME_CONFIG = /etc/apps/x12/local/datetime.xml
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_PREFIX = ~GS\*(?:.*?\*){3}

Here is what datetime.xml ended up looking like:

<datetime>

<define name="_year" extract="year">
    <text><![CDATA[(20\d\d|19\d\d|[901]\d(?!\d))]]></text>
</define>

<define name="_month" extract="month">
    <text><![CDATA[(0?[1-9]|1[012])(?!:)]]></text>
</define>

<define name="_day"  extract="day">
    <text><![CDATA[(0?[1-9]|[12]\d|3[01])]]></text> 
</define>

<define name="_hour" extract="hour">
    <text><![CDATA[([01]?[1-9]|[012][0-3])(?!\d)]]></text>
</define>

<define name="_minute" extract="minute">
    <text><![CDATA[([0-6]\d)(?!\d)]]></text>
</define>

<define name="_second" extract="second">
    <text><![CDATA[([0-6]\d)(?!\d)]]></text>
</define>

<define name="_x12date1" extract="year, month, day, hour, minute, second">
    <text><![CDATA[(19\d\d|20\d\d)(0?[1-9]|1[012])(0[1-9]|[12]\d|3[01])\*(\d{2})(\d{2})(\d{2})]]></text>
</define>

<define name="_x12date2" extract="year, month, day, hour, minute">
    <text><![CDATA[(19\d\d|20\d\d)(0?[1-9]|1[012])(0[1-9]|[12]\d|3[01])\*(\d{2})(\d{2})]]></text>
</define>

<timePatterns>
      <use name="_x12date1"/>
      <use name="_x12date2"/>
</timePatterns>
<datePatterns>
      <use name="_x12date1"/>
      <use name="_x12date2"/>
</datePatterns>

</datetime>

View solution in original post

Path Finder

Hogan24, I am very impressed with your efforts to parse EDI X12. This appears to be a 270. Have you been successful at parsing the full transaction? Have you worked with any other HIPAA Transactions, like the 271 or the 276/277? This solution is the only hit I got regarding EDI/X12 in all of Splunkland. Can you direct me to guidance? Pretty-Please???

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!