All Apps and Add-ons

Qualys Technology Add-on (TA) for Splunk: How can I get the full knowledge base downloaded from Qualys onto my search heads?

Robbie1194
Communicator

Hi guys,

I'm trying to get the full knowledge base downloaded from qualys onto my search heads. I currently have the "basic" knowledge base being downloaded fine. However, I don't know which parameters in the Python scripts to change to download the full knowledge base, I've tried changing detail = Basic to All and the new full knowledgebase is downloaded into the tmp file, however, it's not parsed into the qualys_kb.csv file. I've looked at the logs and the errors I'm seeing look like this:

Exception while parsing. dict contains fields not in fieldnames: 'CVSS_REPORT_CONFIDENCE', 'CVSS_ACCESS', 'CVSS_REMEDIATION_LEVEL', 'CVSS_EXPLOITABILITY', 'CVSS_IMPACT', 'CVSS_AUTHENTICATION' :: Traceback (most recent call last):
    return self.writer.writerow(self._dict_to_list(rowdict))
  File "/opt/splunk/lib/python2.7/csv.py", line 148, in _dict_to_list
ValueError: dict contains fields not in fieldnames: 'CVSS_REPORT_CONFIDENCE', 'CVSS_ACCESS', 'CVSS_REMEDIATION_LEVEL', 'CVSS_EXPLOITABILITY', 'CVSS_IMPACT', 'CVSS_AUTHENTICATION'
TA-QualysCloudPlatform: 2017-09-22T13:18:23Z PID=1552 [MainThread] DEBUG: TA-QualysCloudPlatform [knowledge_base] - Exception while parsing. dict contains fields not in fieldnames: 'CVSS_AUTHENTICATION', 'CVSS_REMEDIATION_LEVEL', 'CVSS_IMPACT', 'CVSS_REPORT_CONFIDENCE', 'CVSS_EXPLOITABILITY', 'CVSS_ACCESS' :: Traceback (most recent call last):
    return self.writer.writerow(self._dict_to_list(rowdict))
  File "/opt/splunk/lib/python2.7/csv.py", line 148, in _dict_to_list
ValueError: dict contains fields not in fieldnames: 'CVSS_AUTHENTICATION', 'CVSS_REMEDIATION_LEVEL', 'CVSS_IMPACT', 'CVSS_REPORT_CONFIDENCE', 'CVSS_EXPLOITABILITY', 'CVSS_ACCESS'

I've looked at the full knowledgebase XML file in the temp directory and it does not contain any of these fieldnames mentioned above (nor does the basic version) so I'm unsure where it's coming from?

Any python wizards out there that can help find what i need to change in the script for the full pull?

I've not explained it too well so if anyone needs more info I'll try and explain it a bit better!

Cheers!

1 Solution

xavierashe
Contributor

Okay, I figured out the trick. As @nit123 mentioned, we have to edit $Splunk_Home/etc/apps/TA-QualysCloudPlatform/bin/qualysModule/splunkpopulator/kbpopulator.py. Around line 30 look for a line that starts with QID_EXTRA_FIELDS_TO_LOG. Edit it to read:

QID_EXTRA_FIELDS_TO_LOG = ["VULN_TYPE", "PATCHABLE", "PCI_FLAG", "TITLE", "CATEGORY", "DIAGNOSIS", "CONSEQUENCE", "SOLUTION", "PUBLISHED_DATETIME", "CVSS_EXPLOITABILITY", "CVSS_REPORT_CONFIDENCE", "CVSS_IMPACT", "CVSS_ACCESS", "CVSS_REMEDIATION_LEVEL", "CVSS_AUTHENTICATION"]

Go down to around line 56. Change "Basic" to "All".

def get_api_parameters(self):
    params = dict(action="list", details="All")
    return params

Before restarting Splunk, delete $Splunk_Home/var/lib/splunk/modinputs/qualys/knowledge_base.

Now, when I did this the first time, I got this error:

TA-QualysCloudPlatform: 2017-11-15T17:52:42Z PID=31979 [MainThread] ERROR: TA-QualysCloudPlatform [knowledge_base] - could not load API response

Being the end of the day, I packed up an drive home. Once I got back online and checked again, it was working fine. I hope this works for you!

View solution in original post

pleymort
Explorer

Hi,

Very useful thank you.
We do the same to get an alert each time an exploit is available on our "internet-facing" assets.
Good way to prioritize patching.

I don't know what the difference in doing that or subscribing to the the Qualys "service" Threat Protection... ?

Regards,
Thibaut

0 Karma

xavierashe
Contributor

Okay, I figured out the trick. As @nit123 mentioned, we have to edit $Splunk_Home/etc/apps/TA-QualysCloudPlatform/bin/qualysModule/splunkpopulator/kbpopulator.py. Around line 30 look for a line that starts with QID_EXTRA_FIELDS_TO_LOG. Edit it to read:

QID_EXTRA_FIELDS_TO_LOG = ["VULN_TYPE", "PATCHABLE", "PCI_FLAG", "TITLE", "CATEGORY", "DIAGNOSIS", "CONSEQUENCE", "SOLUTION", "PUBLISHED_DATETIME", "CVSS_EXPLOITABILITY", "CVSS_REPORT_CONFIDENCE", "CVSS_IMPACT", "CVSS_ACCESS", "CVSS_REMEDIATION_LEVEL", "CVSS_AUTHENTICATION"]

Go down to around line 56. Change "Basic" to "All".

def get_api_parameters(self):
    params = dict(action="list", details="All")
    return params

Before restarting Splunk, delete $Splunk_Home/var/lib/splunk/modinputs/qualys/knowledge_base.

Now, when I did this the first time, I got this error:

TA-QualysCloudPlatform: 2017-11-15T17:52:42Z PID=31979 [MainThread] ERROR: TA-QualysCloudPlatform [knowledge_base] - could not load API response

Being the end of the day, I packed up an drive home. Once I got back online and checked again, it was working fine. I hope this works for you!

xavierashe
Contributor

I am tackling the same problem right now. I know a bit of python so hopefully, I can post an answer soon.

0 Karma

nit123
Path Finder

Robbie ,

Did you try this ?

  1. Open and Edit file kbpopulator python script located under /etc/apps//bin/qualysModule/splunkpopulator/
  2. Check for get_api_parameters function. Change details="Basic" to details="All"
  3. Restart Splunk and check your KB when the inputs run the next time.

Let me know if this helps. Thanks

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...