LINE_BREAKER regex for </issue><issue>

[astackpole]

I have XML files I'm trying to break-up into individual events based on the following XML format. I need to break these up based on the info between each <issue> and </issue>. Also, these <issue>'s are not the top-level XML tag, which I think is causing my issues.

  <issue>
        <finding info>
        <more info, etc. etc.>
  </issue>
  <issue>
      next finding info
  </issue>
  <issue>
    third finding info
  </issue>

I've been trying to use the following with no luck.

[xml-2]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = false
LINE_BREAKER = \s\s(<\/issue>)\s\s\s(<issue>)

 I've also tried BREAK_ONLY_BEFORE as well as KV_MODE = xml with the above configuration.

Any help on this would be GREATLY appreciated!

[amiftah_splunk · Accepted solution]

So with this exact props, you should be able to break events by issue tag:

[ your_srctype]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)\s*\<issue\>
CHARSET=UTF-8
DATETIME_CONFIG=CURRENT

[amiftah_splunk]

Hi @astackpole ,

Have you tried:

LINE_BREAKER = ([\r\n]+)\s*\<issue\>

[astackpole]

I just did and am not seeing much of a difference in results.

Is there a way to ensure it matches exactly to <issue></issue>? Within the tags are different tags also including the word issue, which I think may be causing the problem. Below shows one full event, which should be multiple events.

<issues exportTime="Tue Oct 13 22:44:25 GMT 2020">
  <issue>
    <serialNumber>7183386942052660224</serialNumber>
    <type>5244672</type>
    <name><![CDATA[Session token in URL]]></name>
    <host ip="0.1.2.3">http://0.1.2.3</host>
    <path><![CDATA[/vulnerabilities/csrf/]]></path>
    <location><![CDATA[/vulnerabilities/csrf/]]></location>
    <severity>Medium</severity>
    <confidence>Firm</confidence>
    <issueBackground><![CDATA[<p>Sensitive information within URLs may be logged in various locations, including the user's browser, the web server, and any forward or reverse proxy servers between the two endpoints. URLs may also be displayed on-screen, bookmarked or emailed around by users. They may be disclosed to third parties via the Referer header when any off-site links are followed. Placing session tokens into the URL increases the risk that they will be captured by an attacker.</p>]]></issueBackground>
    <remediationBackground><![CDATA[<p>Applications should use an alternative mechanism for transmitting session tokens, such as HTTP cookies or hidden fields in forms that are submitted using the POST method.</p>]]></remediationBackground>
    <vulnerabilityClassifications><![CDATA[<ul>
<li><a href="https://cwe.mitre.org/data/definitions/200.html">CWE-200: Information Exposure</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/384.html">CWE-384: Session Fixation</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/598.html">CWE-598: Information Exposure Through Query Strings in GET Request</a></li>
</ul>]]></vulnerabilityClassifications>
    <issueDetail><![CDATA[The URL in the request appears to contain a session token within the query string:<ul><li>http://0.1.2.3/vulnerabilities/csrf/?password_current=password&amp;password_new=password&amp;password_conf=password&amp;Change=Change&amp;user_token=126d0ea7389c980cb3ba9872f77f2851</li></ul>]]></issueDetail>
    <issueDetailItems>
      <issueDetailItem><![CDATA[http://0.1.2.3/vulnerabilities/csrf/?password_current=password&password_new=password&password_conf=password&Change=Change&user_token=126d0ea7389c980cb3ba9872f77f2851]]></issueDetailItem>
    </issueDetailItems>
    <requestresponse>
      <request method="GET" base64="true"><![CDATA[BASE64 STRING]]></request>
      <response base64="true"><![CDATA[BASE64>
      <responseRedirected>false</responseRedirected>
    </requestresponse>
  </issue>
  <issue>
    <serialNumber>8062343294186323968</serialNumber>
    <type>5244672</type>
    <name><![CDATA[Session token in URL]]></name>
    <host ip="0.1.2.3">http://0.1.2.3</host>
    <path><![CDATA[/vulnerabilities/xss_r/]]></path>
    <location><![CDATA[/vulnerabilities/xss_r/]]></location>
    <severity>Medium</severity>
    <confidence>Firm</confidence>
    <issueBackground><![CDATA[<p>Sensitive information within URLs may be logged in various locations, including the user's browser, the web server, and any forward or reverse proxy servers between the two endpoints. URLs may also be displayed on-screen, bookmarked or emailed around by users. They may be disclosed to third parties via the Referer header when any off-site links are followed. Placing session tokens into the URL increases the risk that they will be captured by an attacker.</p>]]></issueBackground>
    <remediationBackground><![CDATA[<p>Applications should use an alternative mechanism for transmitting session tokens, such as HTTP cookies or hidden fields in forms that are submitted using the POST method.</p>]]></remediationBackground>
    <vulnerabilityClassifications><![CDATA[<ul>
<li><a href="https://cwe.mitre.org/data/definitions/200.html">CWE-200: Information Exposure</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/384.html">CWE-384: Session Fixation</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/598.html">CWE-598: Information Exposure Through Query Strings in GET Request</a></li>
</ul>]]></vulnerabilityClassifications>
    <issueDetail><![CDATA[The URL in the request appears to contain a session token within the query string:<ul><li>http://0.1.2.3/vulnerabilities/xss_r/?name=nXfPJPjp&amp;user_token=7137bd1753e1e9b101ebc483f3f1c703</li></ul>]]></issueDetail>
    <issueDetailItems>
      <issueDetailItem><![CDATA[http://0.1.2.3/vulnerabilities/xss_r/?name=nXfPJPjp&user_token=7137bd1753e1e9b101ebc483f3f1c703]]></issueDetailItem>
    </issueDetailItems>
  </issue>
  <issue>
    <serialNumber>8136360890162158592</serialNumber>
    <type>5244672</type>
    <name><![CDATA[Session token in URL]]></name>
    <host ip="0.1.2.3">http://0.1.2.3</host>
    <path><![CDATA[/vulnerabilities/xss_r/]]></path>
    <location><![CDATA[/vulnerabilities/xss_r/]]></location>
    <severity>Medium</severity>
    <confidence>Firm</confidence>
    <issueBackground><![CDATA[<p>Sensitive information within URLs may be logged in various locations, including the user's browser, the web server, and any forward or reverse proxy servers between the two endpoints. URLs may also be displayed on-screen, bookmarked or emailed around by users. They may be disclosed to third parties via the Referer header when any off-site links are followed. Placing session tokens into the URL increases the risk that they will be captured by an attacker.</p>]]></issueBackground>
    <remediationBackground><![CDATA[<p>Applications should use an alternative mechanism for transmitting session tokens, such as HTTP cookies or hidden fields in forms that are submitted using the POST method.</p>]]></remediationBackground>
    <vulnerabilityClassifications><![CDATA[<ul>
<li><a href="https://cwe.mitre.org/data/definitions/200.html">CWE-200: Information Exposure</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/384.html">CWE-384: Session Fixation</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/598.html">CWE-598: Information Exposure Through Query Strings in GET Request</a></li>
</ul>]]></vulnerabilityClassifications>
    <issueDetail><![CDATA[The URL in the request appears to contain a session token within the query string:<ul><li>http://0.1.2.3/vulnerabilities/xss_r/?name=MbMaFuRM&amp;user_token=64dc95dcaa45bb1162514d0a9285b200</li></ul>]]></issueDetail>
    <issueDetailItems>
    </issueDetailItem>
    </issueDetailItems>
    <requestresponse>
      <request method="GET" base64="true">
      <responseRedirected>false</responseRedirected>
    </requestresponse>
  </issue>
  <issue>
    <serialNumber>4136405686680984576</serialNumber>
    <type>5244672</type>
    <name><![CDATA[Session token in URL]]></name>
    <host ip="0.1.2.3">http://0.1.2.3</host>
    <path><![CDATA[/vulnerabilities/sqli_blind/]]></path>
    <location><![CDATA[/vulnerabilities/sqli_blind/]]></location>
    <severity>Medium</severity>
    <confidence>Firm</confidence>
    <issueBackground><![CDATA[<p>Sensitive information within URLs may be logged in various locations, including the user's browser, the web server, and any forward or reverse proxy servers between the two endpoints. URLs may also be displayed on-screen, bookmarked or emailed around by users. They may be disclosed to third parties via the Referer header when any off-site links are followed. Placing session tokens into the URL increases the risk that they will be captured by an attacker.</p>]]></issueBackground>
    <remediationBackground><![CDATA[<p>Applications should use an alternative mechanism for transmitting session tokens, such as HTTP cookies or hidden fields in forms that are submitted using the POST method.</p>]]></remediationBackground>
    <vulnerabilityClassifications><![CDATA[<ul>
<li><a href="https://cwe.mitre.org/data/definitions/200.html">CWE-200: Information Exposure</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/384.html">CWE-384: Session Fixation</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/598.html">CWE-598: Information Exposure Through Query Strings in GET Request</a></li>
</ul>]]></vulnerabilityClassifications>
    <issueDetail><![CDATA[The URL in the request appears to contain a session token within the query string:<ul><li>http://0.1.2.3/vulnerabilities/sqli_blind/?id=185006&amp;Submit=Submit&amp;user_token=40edd90de7cbcb3775313c679511c516</li></ul>]]></issueDetail>
    <issueDetailItems>
      <issueDetailItem><![CDATA[http://0.1.2.3/vulnerabilities/sqli_blind/?id=185006&Submit=Submit&user_token=40edd90de7cbcb3775313c679511c516]]></issueDetailItem>
    </issueDetailItems>
    <requestresponse>
      <request method="GET" base64="true"><![CDATA[BASE64 STRING]]></request>
      <response base64="true"><![CDATA[BASE64 STRING]]></response>
      <responseRedirected>false</responseRedirected>
    </requestresponse>
  </issue>
  <issue>
    <serialNumber>8478452543278705664</serialNumber>
    <type>5244672</type>
    <name><![CDATA[Session token in URL]]></name>
    <host ip="0.1.2.3">http://0.1.2.3</host>
    <path><![CDATA[/vulnerabilities/sqli/]]></path>
    <location><![CDATA[/vulnerabilities/sqli/]]></location>
    <severity>Medium</severity>
    <confidence>Firm</confidence>
    <issueBackground><![CDATA[<p>Sensitive information within URLs may be logged in various locations, including the user's browser, the web server, and any forward or reverse proxy servers between the two endpoints. URLs may also be displayed on-screen, bookmarked or emailed around by users. They may be disclosed to third parties via the Referer header when any off-site links are followed. Placing session tokens into the URL increases the risk that they will be captured by an attacker.</p>]]></issueBackground>
    <remediationBackground><![CDATA[<p>Applications should use an alternative mechanism for transmitting session tokens, such as HTTP cookies or hidden fields in forms that are submitted using the POST method.</p>]]></remediationBackground>
    <vulnerabilityClassifications><![CDATA[<ul>
<li><a href="https://cwe.mitre.org/data/definitions/200.html">CWE-200: Information Exposure</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/384.html">CWE-384: Session Fixation</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/598.html">CWE-598: Information Exposure Through Query Strings in GET Request</a></li>
</ul>]]></vulnerabilityClassifications>
    <issueDetail><![CDATA[The URL in the request appears to contain a session token within the query string:<ul><li>http://0.1.2.3/vulnerabilities/sqli/?id=872225&amp;Submit=Submit&amp;user_token=84b74af6bd7645aa202c7f370b7ef6a8</li></ul>]]></issueDetail>
    <issueDetailItems>
      <issueDetailItem><![CDATA[http://0.1.2.3/vulnerabilities/sqli/?id=872225&Submit=Submit&user_token=84b74af6bd7645aa202c7f370b7ef6a8]]></issueDetailItem>
    </issueDetailItems>
    <requestresponse>
      <request method="GET" base64="true"><![CDATA[BASE64 STRING]]></response>
      <responseRedirected>false</responseRedirected>
    </requestresponse>
  </issue>
  <issue>
    <serialNumber>4828258302681230336</serialNumber>
    <type>5244672</type>
    <name><![CDATA[Session token in URL]]></name>
    <host ip="0.1.2.3">http://0.1.2.3</host>
    <path><![CDATA[/vulnerabilities/csrf/]]></path>
    <location><![CDATA[/vulnerabilities/csrf/]]></location>
    <severity>Medium</severity>
    <confidence>Firm</confidence>
    <issueBackground><![CDATA[<p>Sensitive information within URLs may be logged in various locations, including the user's browser, the web server, and any forward or reverse proxy servers between the two endpoints. URLs may also be displayed on-screen, bookmarked or emailed around by users. They may be disclosed to third parties via the Referer header when any off-site links are followed. Placing session tokens into the URL increases the risk that they will be captured by an attacker.</p>]]></issueBackground>
    <remediationBackground><![CDATA[<p>Applications should use an alternative mechanism for transmitting session tokens, such as HTTP cookies or hidden fields in forms that are submitted using the POST method.</p>]]></remediationBackground>
    <vulnerabilityClassifications><![CDATA[<ul>
<li><a href="https://cwe.mitre.org/data/definitions/200.html">CWE-200: Information Exposure</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/384.html">CWE-384: Session Fixation</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/598.html">CWE-598: Information Exposure Through Query Strings in GET Request</a></li>
</ul>]]></vulnerabilityClassifications>
    <issueDetail><![CDATA[The URL in the request appears to contain a session token within the query string:<ul><li>http://0.1.2.3/vulnerabilities/csrf/?password_current=password&amp;password_new=password&amp;password_conf=password&amp;Change=Change&amp;user_token=cd07548bd2d7109e5e57b4e9830bbcee</li></ul>]]></issueDetail>
    <issueDetailItems>
      <issueDetailItem><![CDATA[http://0.0.0.0/vulnerabilities/csrf/?password_current=password&password_new=password&password_conf=password&Change=Change&user_token=cd07548bd2d7109e5e57b4e9830bbcee]]></issueDetailItem>
    </issueDetailItems>
    <requestresponse>
      <request method="GET" base64="true"><![CDATA[BASE64 DATA]]></response>
      <responseRedirected>false</responseRedirected>
    </requestresponse>
  </issue>
</issues>