Qualys Technology Add-on (TA) for Splunk 1.0.3: Why am I getting "Error -5 while decompressing data: incomplete or truncated stream"?

[todd_miller]

Since updating to version 1.0.3 of the Qualys Technology Add-on (TA) for Splunk (Running on a dedicated "API Forwarder", a standalone Splunk 6.4.0 instance that forwards data to my indexers), I can no longer ingest data. On version 1.0.2, I was only getting the scan data, no KB data).

Here is the error I'm getting:

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualys.py" Traceback (most recent call last):

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualys.py"   File "/opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualys.py", line 274, in <module>

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualys.py"     main()

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualys.py"   File "/opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualys.py", line 267, in main

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualys.py"     run()

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualys.py"   File "/opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualys.py", line 144, in run

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualys.py"     api_password = qualysModule.splunkpopulator.utils.decrypt(qualysConf['setupentity']['password'])

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualys.py"   File "/opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualysModule/splunkpopulator/utils.py", line 201, in decrypt

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualys.py"     return zlib.decompress(base64.urlsafe_b64decode(text))

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-QualysCloudPlatform/bin/qualys.py" zlib.error: Error -5 while decompressing data: incomplete or truncated stream

Any additional insight would be appreciated!

[todd_miller]

So the good news is that I don't appear to be seeing any errors anymore.

The bad news is the following:

• When "knowledge_base" and "host_detection" data inputs are enabled, only "knowledge_base" data seems to be downloaded
• I see a tmp file created with "knowledge_base" data. Total file size is ~125M.
• The timestamp on the qualys_kb.csv lookup file is updated but no additional data is added to it. Current file size is 5.3M. I also created an empty file and added the csv header information to it. It successful recreates a 5.3M file

• When the "knowledge_base" data input is disabled, nothing is downloaded from the "host_detection" data input. I see the following in the logs.

  5/11/16
  9:14:07.000 AM  
  making https://qualysapi.qualys.com/msp/about.php request with params={}
  host = x.x.com index = _internal source = qualys://host_detection sourcetype = qualys
  
  5/11/16
  9:14:07.000 AM  
  Start qualys TA
  host = x.x.com index = _internal source = qualys://host_detection sourcetype = qualys
  
  5/11/16
  9:14:04.000 AM  
  End qualys TA
  host = x.x.com index = _internal source = qualys://host_detection sourcetype = qualys
  

That's all we get.

[jeffriesa]

I have just upgraded away from the Beta version which was working fine.

We are getting the same errors?

And this one:
QualysSplunkPopulator: 2016-08-11T11:14:21Z PID=25745 [MainThread] ERROR: QualysSplunkPopulator - Error during request to /msp/about.php, [None] Unauthorized

[Lindaiyu]

Hello Todd,

Does your app work now? If not, you could communicate with me.