CVSS VECTOR STRING field error Qualys TA

[kaw243]

Has anyone had an issue with the qualys-technology-add-on-ta-for-splunk_134 TA that started today? It looks like the API is pulling a new field VECTOR_STRING and the parsing script can not handle it.

This is an example of the same QID from 12/18/2018

      <BASE source="service">5</BASE>
      <TEMPORAL>3.6</TEMPORAL>
    </CVSS>
    <PCI_FLAG>1</PCI_FLAG>
    <THREAT_INTELLIGENCE>
      <THREAT_INTEL id="5"><![CDATA[Easy_Exploit]]></THREAT_INTEL>
      <THREAT_INTEL id="8"><![CDATA[No_Patch]]></THREAT_INTEL>
    </THREAT_INTELLIGENCE>
    <DISCOVERY>
      <REMOTE>1</REMOTE>
    </DISCOVERY>
  </VULN>
  <VULN>
    <QID>32</QID>'

Here is the one from today

      <BASE source="service">5</BASE>
      <TEMPORAL>3.6</TEMPORAL>
      **<VECTOR_STRING>CVSS:2.0/AV:N/AC:L/Au:N/C:P/I:N/A:N/E:U/RL:W/RC:UC</VECTOR_STRING>**
    </CVSS>
    <PCI_FLAG>1</PCI_FLAG>
    <THREAT_INTELLIGENCE>
      <THREAT_INTEL id="5"><![CDATA[Easy_Exploit]]></THREAT_INTEL>
      <THREAT_INTEL id="8"><![CDATA[No_Patch]]></THREAT_INTEL>
    </THREAT_INTELLIGENCE>
    <DISCOVERY>
      <REMOTE>1</REMOTE>
    </DISCOVERY>
  </VULN>
  <VULN>
    <QID>32</QID>

Here are the error logs

TA-QualysCloudPlatform: 2018-12-20 12:35:54 PID=68179 [MainThread] ERROR: TA-QualysCloudPlatform (knowledge_base) - could not load API response. Reason: dict contains fields not in fieldnames: 'CVSS_VECTOR_STRING'
ValueError: dict contains fields not in fieldnames: 'CVSS_VECTOR_STRING'
+ ", ".join([repr(x) for x in wrong_fields]))
File "/splunk/apps/splunk/lib/python2.7/csv.py", line 148, in _dict_to_list
return self.writer.writerow(self._dict_to_list(rowdict))
File "/splunk/apps/splunk/lib/python2.7/csv.py", line 152, in writerow
writer.writerow(self._qids[qid])
File "/splunk/apps/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualysModule/splunkpopulator/kbpopulator.py", line 133, in _post_parse
self._post_parse()
File "/splunk/apps/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualysModule/splunkpopulator/basepopulator.py", line 287, in _parse
parseresponse = self._parse(response.file_name)
File "/splunk/apps/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualysModule/splunkpopulator/basepopulator.py", line 134, in __fetch_and_parse
TA-QualysCloudPlatform: 2018-12-20 12:35:54 PID=68179 [MainThread] DEBUG: TA-QualysCloudPlatform (knowledge_base) - Exception while parsing. dict contains fields not in fieldnames: 'CVSS_VECTOR_STRING' :: Traceback (most recent call last):

[tonylieu70 · Accepted solution]

Solution:
Go to the kbpopulator.py file and change this line of code to include the new variable CVSS_VECTOR_STRING

CVS_HEADER_COLUMNS = ["QID", "SEVERITY"] + QID_EXTRA_FIELDS_TO_LOG + ["CVSS_BASE", "CVSS_TEMPORAL", "CVSS_VECTOR_STRING", "CVE", "VENDOR_REFERENCE"]
def init(self, logger=None):

If your script is a little different, double check the script to see if there are any restrictions on variables or any variable references you need to change.

[martin_mueller]

You can make your lookup more robust by moving the KB download out to the heavy forwarder that does your host detection download, batch-input the lookup from there into your indexers, and update your lookup on the SH(s) via scheduled searches that can be as robust as you like.

[martin_mueller]

As a note to the Qualys TA devs: Even after this fix the underlying bug is looming, waiting to pounce again.

The CVSS-sub-element-reading-code will read any sub-element and create CSV cells from that... however, your CSV headers are static. Either stop reading fields dynamically, or create the CSV header dynamically after reading all fields for all QIDs.

[andygerberkp]

The code contains several references to a person with email "bharrat@gmail.com"; I've emailed him with this comment as well. (If anyone from Qualys reads this).

The problem I see in the current
add-on is it overwrites the
qualys_kb.csv lookup file with a zero
length file on failure. It would make
more sense if it would abort and keep
the original copy on failure, or at
least make a backup.

[harsmarvania57]

Hi,

It looks like Qualys added this new field in API but forgot to add parsing in Add-on. I am seeing same behavior with version 1.3.3

[andygerberkp]

Also a problem with 1.2.3. Solution mentioned below fixes it fine.