Getting Data In

LINE_BREAKER regex for </issue><issue>

astackpole
Path Finder

I have XML files I'm trying to break-up into individual events based on the following XML format. I need to break these up based on the info between each <issue> and </issue>. Also, these <issue>'s are not the top-level XML tag, which I think is causing my issues.

  <issue>
        <finding info>
        <more info, etc. etc.>
  </issue>
  <issue>
      next finding info
  </issue>
  <issue>
    third finding info
  </issue>

I've been trying to use the following with no luck.

[xml-2]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = false
LINE_BREAKER = \s\s(<\/issue>)\s\s\s(<issue>)

 I've also tried BREAK_ONLY_BEFORE as well as KV_MODE = xml with the above configuration.

Any help on this would be GREATLY appreciated!

Labels (1)
Tags (2)
0 Karma
1 Solution

amiftah_splunk
Splunk Employee
Splunk Employee

So with this exact props, you should be able to break events by issue tag:

[ your_srctype]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)\s*\<issue\>
CHARSET=UTF-8
DATETIME_CONFIG=CURRENT

View solution in original post

amiftah_splunk
Splunk Employee
Splunk Employee

So with this exact props, you should be able to break events by issue tag:

[ your_srctype]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)\s*\<issue\>
CHARSET=UTF-8
DATETIME_CONFIG=CURRENT

amiftah_splunk
Splunk Employee
Splunk Employee

Hi @astackpole ,

Have you tried:

LINE_BREAKER = ([\r\n]+)\s*\<issue\>

 

0 Karma

astackpole
Path Finder

I just did and am not seeing much of a difference in results.

Is there a way to ensure it matches exactly to <issue></issue>? Within the tags are different tags also including the word issue, which I think may be causing the problem. Below shows one full event, which should be multiple events.

 

<issues exportTime="Tue Oct 13 22:44:25 GMT 2020">
  <issue>
    <serialNumber>7183386942052660224</serialNumber>
    <type>5244672</type>
    <name><![CDATA[Session token in URL]]></name>
    <host ip="0.1.2.3">http://0.1.2.3</host>
    <path><![CDATA[/vulnerabilities/csrf/]]></path>
    <location><![CDATA[/vulnerabilities/csrf/]]></location>
    <severity>Medium</severity>
    <confidence>Firm</confidence>
    <issueBackground><![CDATA[<p>Sensitive information within URLs may be logged in various locations, including the user's browser, the web server, and any forward or reverse proxy servers between the two endpoints. URLs may also be displayed on-screen, bookmarked or emailed around by users. They may be disclosed to third parties via the Referer header when any off-site links are followed. Placing session tokens into the URL increases the risk that they will be captured by an attacker.</p>]]></issueBackground>
    <remediationBackground><![CDATA[<p>Applications should use an alternative mechanism for transmitting session tokens, such as HTTP cookies or hidden fields in forms that are submitted using the POST method.</p>]]></remediationBackground>
    <vulnerabilityClassifications><![CDATA[<ul>
<li><a href="https://cwe.mitre.org/data/definitions/200.html">CWE-200: Information Exposure</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/384.html">CWE-384: Session Fixation</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/598.html">CWE-598: Information Exposure Through Query Strings in GET Request</a></li>
</ul>]]></vulnerabilityClassifications>
    <issueDetail><![CDATA[The URL in the request appears to contain a session token within the query string:<ul><li>http://0.1.2.3/vulnerabilities/csrf/?password_current=password&amp;password_new=password&amp;password_conf=password&amp;Change=Change&amp;user_token=126d0ea7389c980cb3ba9872f77f2851</li></ul>]]></issueDetail>
    <issueDetailItems>
      <issueDetailItem><![CDATA[http://0.1.2.3/vulnerabilities/csrf/?password_current=password&password_new=password&password_conf=password&Change=Change&user_token=126d0ea7389c980cb3ba9872f77f2851]]></issueDetailItem>
    </issueDetailItems>
    <requestresponse>
      <request method="GET" base64="true"><![CDATA[BASE64 STRING]]></request>
      <response base64="true"><![CDATA[BASE64>
      <responseRedirected>false</responseRedirected>
    </requestresponse>
  </issue>
  <issue>
    <serialNumber>8062343294186323968</serialNumber>
    <type>5244672</type>
    <name><![CDATA[Session token in URL]]></name>
    <host ip="0.1.2.3">http://0.1.2.3</host>
    <path><![CDATA[/vulnerabilities/xss_r/]]></path>
    <location><![CDATA[/vulnerabilities/xss_r/]]></location>
    <severity>Medium</severity>
    <confidence>Firm</confidence>
    <issueBackground><![CDATA[<p>Sensitive information within URLs may be logged in various locations, including the user's browser, the web server, and any forward or reverse proxy servers between the two endpoints. URLs may also be displayed on-screen, bookmarked or emailed around by users. They may be disclosed to third parties via the Referer header when any off-site links are followed. Placing session tokens into the URL increases the risk that they will be captured by an attacker.</p>]]></issueBackground>
    <remediationBackground><![CDATA[<p>Applications should use an alternative mechanism for transmitting session tokens, such as HTTP cookies or hidden fields in forms that are submitted using the POST method.</p>]]></remediationBackground>
    <vulnerabilityClassifications><![CDATA[<ul>
<li><a href="https://cwe.mitre.org/data/definitions/200.html">CWE-200: Information Exposure</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/384.html">CWE-384: Session Fixation</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/598.html">CWE-598: Information Exposure Through Query Strings in GET Request</a></li>
</ul>]]></vulnerabilityClassifications>
    <issueDetail><![CDATA[The URL in the request appears to contain a session token within the query string:<ul><li>http://0.1.2.3/vulnerabilities/xss_r/?name=nXfPJPjp&amp;user_token=7137bd1753e1e9b101ebc483f3f1c703</li></ul>]]></issueDetail>
    <issueDetailItems>
      <issueDetailItem><![CDATA[http://0.1.2.3/vulnerabilities/xss_r/?name=nXfPJPjp&user_token=7137bd1753e1e9b101ebc483f3f1c703]]></issueDetailItem>
    </issueDetailItems>
  </issue>
  <issue>
    <serialNumber>8136360890162158592</serialNumber>
    <type>5244672</type>
    <name><![CDATA[Session token in URL]]></name>
    <host ip="0.1.2.3">http://0.1.2.3</host>
    <path><![CDATA[/vulnerabilities/xss_r/]]></path>
    <location><![CDATA[/vulnerabilities/xss_r/]]></location>
    <severity>Medium</severity>
    <confidence>Firm</confidence>
    <issueBackground><![CDATA[<p>Sensitive information within URLs may be logged in various locations, including the user's browser, the web server, and any forward or reverse proxy servers between the two endpoints. URLs may also be displayed on-screen, bookmarked or emailed around by users. They may be disclosed to third parties via the Referer header when any off-site links are followed. Placing session tokens into the URL increases the risk that they will be captured by an attacker.</p>]]></issueBackground>
    <remediationBackground><![CDATA[<p>Applications should use an alternative mechanism for transmitting session tokens, such as HTTP cookies or hidden fields in forms that are submitted using the POST method.</p>]]></remediationBackground>
    <vulnerabilityClassifications><![CDATA[<ul>
<li><a href="https://cwe.mitre.org/data/definitions/200.html">CWE-200: Information Exposure</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/384.html">CWE-384: Session Fixation</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/598.html">CWE-598: Information Exposure Through Query Strings in GET Request</a></li>
</ul>]]></vulnerabilityClassifications>
    <issueDetail><![CDATA[The URL in the request appears to contain a session token within the query string:<ul><li>http://0.1.2.3/vulnerabilities/xss_r/?name=MbMaFuRM&amp;user_token=64dc95dcaa45bb1162514d0a9285b200</li></ul>]]></issueDetail>
    <issueDetailItems>
    </issueDetailItem>
    </issueDetailItems>
    <requestresponse>
      <request method="GET" base64="true">
      <responseRedirected>false</responseRedirected>
    </requestresponse>
  </issue>
  <issue>
    <serialNumber>4136405686680984576</serialNumber>
    <type>5244672</type>
    <name><![CDATA[Session token in URL]]></name>
    <host ip="0.1.2.3">http://0.1.2.3</host>
    <path><![CDATA[/vulnerabilities/sqli_blind/]]></path>
    <location><![CDATA[/vulnerabilities/sqli_blind/]]></location>
    <severity>Medium</severity>
    <confidence>Firm</confidence>
    <issueBackground><![CDATA[<p>Sensitive information within URLs may be logged in various locations, including the user's browser, the web server, and any forward or reverse proxy servers between the two endpoints. URLs may also be displayed on-screen, bookmarked or emailed around by users. They may be disclosed to third parties via the Referer header when any off-site links are followed. Placing session tokens into the URL increases the risk that they will be captured by an attacker.</p>]]></issueBackground>
    <remediationBackground><![CDATA[<p>Applications should use an alternative mechanism for transmitting session tokens, such as HTTP cookies or hidden fields in forms that are submitted using the POST method.</p>]]></remediationBackground>
    <vulnerabilityClassifications><![CDATA[<ul>
<li><a href="https://cwe.mitre.org/data/definitions/200.html">CWE-200: Information Exposure</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/384.html">CWE-384: Session Fixation</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/598.html">CWE-598: Information Exposure Through Query Strings in GET Request</a></li>
</ul>]]></vulnerabilityClassifications>
    <issueDetail><![CDATA[The URL in the request appears to contain a session token within the query string:<ul><li>http://0.1.2.3/vulnerabilities/sqli_blind/?id=185006&amp;Submit=Submit&amp;user_token=40edd90de7cbcb3775313c679511c516</li></ul>]]></issueDetail>
    <issueDetailItems>
      <issueDetailItem><![CDATA[http://0.1.2.3/vulnerabilities/sqli_blind/?id=185006&Submit=Submit&user_token=40edd90de7cbcb3775313c679511c516]]></issueDetailItem>
    </issueDetailItems>
    <requestresponse>
      <request method="GET" base64="true"><![CDATA[BASE64 STRING]]></request>
      <response base64="true"><![CDATA[BASE64 STRING]]></response>
      <responseRedirected>false</responseRedirected>
    </requestresponse>
  </issue>
  <issue>
    <serialNumber>8478452543278705664</serialNumber>
    <type>5244672</type>
    <name><![CDATA[Session token in URL]]></name>
    <host ip="0.1.2.3">http://0.1.2.3</host>
    <path><![CDATA[/vulnerabilities/sqli/]]></path>
    <location><![CDATA[/vulnerabilities/sqli/]]></location>
    <severity>Medium</severity>
    <confidence>Firm</confidence>
    <issueBackground><![CDATA[<p>Sensitive information within URLs may be logged in various locations, including the user's browser, the web server, and any forward or reverse proxy servers between the two endpoints. URLs may also be displayed on-screen, bookmarked or emailed around by users. They may be disclosed to third parties via the Referer header when any off-site links are followed. Placing session tokens into the URL increases the risk that they will be captured by an attacker.</p>]]></issueBackground>
    <remediationBackground><![CDATA[<p>Applications should use an alternative mechanism for transmitting session tokens, such as HTTP cookies or hidden fields in forms that are submitted using the POST method.</p>]]></remediationBackground>
    <vulnerabilityClassifications><![CDATA[<ul>
<li><a href="https://cwe.mitre.org/data/definitions/200.html">CWE-200: Information Exposure</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/384.html">CWE-384: Session Fixation</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/598.html">CWE-598: Information Exposure Through Query Strings in GET Request</a></li>
</ul>]]></vulnerabilityClassifications>
    <issueDetail><![CDATA[The URL in the request appears to contain a session token within the query string:<ul><li>http://0.1.2.3/vulnerabilities/sqli/?id=872225&amp;Submit=Submit&amp;user_token=84b74af6bd7645aa202c7f370b7ef6a8</li></ul>]]></issueDetail>
    <issueDetailItems>
      <issueDetailItem><![CDATA[http://0.1.2.3/vulnerabilities/sqli/?id=872225&Submit=Submit&user_token=84b74af6bd7645aa202c7f370b7ef6a8]]></issueDetailItem>
    </issueDetailItems>
    <requestresponse>
      <request method="GET" base64="true"><![CDATA[BASE64 STRING]]></response>
      <responseRedirected>false</responseRedirected>
    </requestresponse>
  </issue>
  <issue>
    <serialNumber>4828258302681230336</serialNumber>
    <type>5244672</type>
    <name><![CDATA[Session token in URL]]></name>
    <host ip="0.1.2.3">http://0.1.2.3</host>
    <path><![CDATA[/vulnerabilities/csrf/]]></path>
    <location><![CDATA[/vulnerabilities/csrf/]]></location>
    <severity>Medium</severity>
    <confidence>Firm</confidence>
    <issueBackground><![CDATA[<p>Sensitive information within URLs may be logged in various locations, including the user's browser, the web server, and any forward or reverse proxy servers between the two endpoints. URLs may also be displayed on-screen, bookmarked or emailed around by users. They may be disclosed to third parties via the Referer header when any off-site links are followed. Placing session tokens into the URL increases the risk that they will be captured by an attacker.</p>]]></issueBackground>
    <remediationBackground><![CDATA[<p>Applications should use an alternative mechanism for transmitting session tokens, such as HTTP cookies or hidden fields in forms that are submitted using the POST method.</p>]]></remediationBackground>
    <vulnerabilityClassifications><![CDATA[<ul>
<li><a href="https://cwe.mitre.org/data/definitions/200.html">CWE-200: Information Exposure</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/384.html">CWE-384: Session Fixation</a></li>
<li><a href="https://cwe.mitre.org/data/definitions/598.html">CWE-598: Information Exposure Through Query Strings in GET Request</a></li>
</ul>]]></vulnerabilityClassifications>
    <issueDetail><![CDATA[The URL in the request appears to contain a session token within the query string:<ul><li>http://0.1.2.3/vulnerabilities/csrf/?password_current=password&amp;password_new=password&amp;password_conf=password&amp;Change=Change&amp;user_token=cd07548bd2d7109e5e57b4e9830bbcee</li></ul>]]></issueDetail>
    <issueDetailItems>
      <issueDetailItem><![CDATA[http://0.0.0.0/vulnerabilities/csrf/?password_current=password&password_new=password&password_conf=password&Change=Change&user_token=cd07548bd2d7109e5e57b4e9830bbcee]]></issueDetailItem>
    </issueDetailItems>
    <requestresponse>
      <request method="GET" base64="true"><![CDATA[BASE64 DATA]]></response>
      <responseRedirected>false</responseRedirected>
    </requestresponse>
  </issue>
</issues>

 

 

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...