Hi guys,
I'm trying to get the full knowledge base downloaded from qualys onto my search heads. I currently have the "basic" knowledge base being downloaded fine. However, I don't know which parameters in the Python scripts to change to download the full knowledge base, I've tried changing detail = Basic to All and the new full knowledgebase is downloaded into the tmp file, however, it's not parsed into the qualys_kb.csv file. I've looked at the logs and the errors I'm seeing look like this:
Exception while parsing. dict contains fields not in fieldnames: 'CVSS_REPORT_CONFIDENCE', 'CVSS_ACCESS', 'CVSS_REMEDIATION_LEVEL', 'CVSS_EXPLOITABILITY', 'CVSS_IMPACT', 'CVSS_AUTHENTICATION' :: Traceback (most recent call last):
return self.writer.writerow(self._dict_to_list(rowdict))
File "/opt/splunk/lib/python2.7/csv.py", line 148, in _dict_to_list
ValueError: dict contains fields not in fieldnames: 'CVSS_REPORT_CONFIDENCE', 'CVSS_ACCESS', 'CVSS_REMEDIATION_LEVEL', 'CVSS_EXPLOITABILITY', 'CVSS_IMPACT', 'CVSS_AUTHENTICATION'
TA-QualysCloudPlatform: 2017-09-22T13:18:23Z PID=1552 [MainThread] DEBUG: TA-QualysCloudPlatform [knowledge_base] - Exception while parsing. dict contains fields not in fieldnames: 'CVSS_AUTHENTICATION', 'CVSS_REMEDIATION_LEVEL', 'CVSS_IMPACT', 'CVSS_REPORT_CONFIDENCE', 'CVSS_EXPLOITABILITY', 'CVSS_ACCESS' :: Traceback (most recent call last):
return self.writer.writerow(self._dict_to_list(rowdict))
File "/opt/splunk/lib/python2.7/csv.py", line 148, in _dict_to_list
ValueError: dict contains fields not in fieldnames: 'CVSS_AUTHENTICATION', 'CVSS_REMEDIATION_LEVEL', 'CVSS_IMPACT', 'CVSS_REPORT_CONFIDENCE', 'CVSS_EXPLOITABILITY', 'CVSS_ACCESS'
I've looked at the full knowledgebase XML file in the temp directory and it does not contain any of these fieldnames mentioned above (nor does the basic version) so I'm unsure where it's coming from?
Any python wizards out there that can help find what i need to change in the script for the full pull?
I've not explained it too well so if anyone needs more info I'll try and explain it a bit better!
Cheers!
Okay, I figured out the trick. As @nit123 mentioned, we have to edit $Splunk_Home/etc/apps/TA-QualysCloudPlatform/bin/qualysModule/splunkpopulator/kbpopulator.py. Around line 30 look for a line that starts with QID_EXTRA_FIELDS_TO_LOG. Edit it to read:
QID_EXTRA_FIELDS_TO_LOG = ["VULN_TYPE", "PATCHABLE", "PCI_FLAG", "TITLE", "CATEGORY", "DIAGNOSIS", "CONSEQUENCE", "SOLUTION", "PUBLISHED_DATETIME", "CVSS_EXPLOITABILITY", "CVSS_REPORT_CONFIDENCE", "CVSS_IMPACT", "CVSS_ACCESS", "CVSS_REMEDIATION_LEVEL", "CVSS_AUTHENTICATION"]
Go down to around line 56. Change "Basic" to "All".
def get_api_parameters(self):
params = dict(action="list", details="All")
return params
Before restarting Splunk, delete $Splunk_Home/var/lib/splunk/modinputs/qualys/knowledge_base.
Now, when I did this the first time, I got this error:
TA-QualysCloudPlatform: 2017-11-15T17:52:42Z PID=31979 [MainThread] ERROR: TA-QualysCloudPlatform [knowledge_base] - could not load API response
Being the end of the day, I packed up an drive home. Once I got back online and checked again, it was working fine. I hope this works for you!
Hi,
Very useful thank you.
We do the same to get an alert each time an exploit is available on our "internet-facing" assets.
Good way to prioritize patching.
I don't know what the difference in doing that or subscribing to the the Qualys "service" Threat Protection... ?
Regards,
Thibaut
Okay, I figured out the trick. As @nit123 mentioned, we have to edit $Splunk_Home/etc/apps/TA-QualysCloudPlatform/bin/qualysModule/splunkpopulator/kbpopulator.py. Around line 30 look for a line that starts with QID_EXTRA_FIELDS_TO_LOG. Edit it to read:
QID_EXTRA_FIELDS_TO_LOG = ["VULN_TYPE", "PATCHABLE", "PCI_FLAG", "TITLE", "CATEGORY", "DIAGNOSIS", "CONSEQUENCE", "SOLUTION", "PUBLISHED_DATETIME", "CVSS_EXPLOITABILITY", "CVSS_REPORT_CONFIDENCE", "CVSS_IMPACT", "CVSS_ACCESS", "CVSS_REMEDIATION_LEVEL", "CVSS_AUTHENTICATION"]
Go down to around line 56. Change "Basic" to "All".
def get_api_parameters(self):
params = dict(action="list", details="All")
return params
Before restarting Splunk, delete $Splunk_Home/var/lib/splunk/modinputs/qualys/knowledge_base.
Now, when I did this the first time, I got this error:
TA-QualysCloudPlatform: 2017-11-15T17:52:42Z PID=31979 [MainThread] ERROR: TA-QualysCloudPlatform [knowledge_base] - could not load API response
Being the end of the day, I packed up an drive home. Once I got back online and checked again, it was working fine. I hope this works for you!
I am tackling the same problem right now. I know a bit of python so hopefully, I can post an answer soon.
Robbie ,
Did you try this ?
Let me know if this helps. Thanks