Qualys TA: WARNING: Failed to parse API Output...XML or text declaration not at start of entity

[ejwade]

My Qualys VM detection pull stopped working. I found a new warning log.

TA-QualysCloudPlatform (host_detection): 2023-03-06 08:54:15 PID=30479 [Thread-3] WARNING: Failed to parse API Output for endpoint /api/2.0/fo/asset/host/vm/detection/. Message: XML or text declaration not at start of entity: line 7, column 0

Has anyone come across this? I have no idea where to start when it comes to troubleshooting.

[Tom_Lundie]

I'm not familiar with this TA, but I've had a read through the code and it looks like there could be a couple of issues at play here. Please take this advice with a pinch of salt because I haven't been able to directly test this TA.

First and foremost: The TA does not seem to raise any exceptions (and interrupt the normal flow) when the API returns a non OK response. It does seem to enter a debug log for non-200 response codes. Can you see "Got NOK response from API" in the previous line(s) of your logs?

if request.getcode() != 200:
    qlogger.debug("Got NOK response from API")

This would indicate that something is wrong on the API-side. These issues tend to be related to authentication/authorization issues. Make sure that your API credentials have not expired and the correct permissions have been set to read host vm_detection's.

If you're happy with API configuration, the next thing that I would look at is the fact that the TA seems to stage the API output in ./TA-QualysCloudPlatform/tmp/ before ingesting it (so $SPLUNK_HOME/etc/apps/TA-QualysCloudPlatform/tmp/) .

Make sure that the volume mounted for this directory is not full up.

df -h

Given that this error corresponds with the fact that the API is not returning data in the expected XML format, I would try and isolate the relevant ./tmp/ file and see if there are any relevant clues such as unusual characters etc. If there are lots of ./tmp/ files in this directory then you could try deleting them to make sure any old half-written files are not kicking around and causing the issue.

These filenames are generated dynamically using properties such as os.getpid() and current_thread().getName() so it's going to be quite hard for the folks on Splunk Community to help you debug this remotely.

That being said, if nothing obvious is jumping out with the above, then I suggest that you reach out to Qualys directly as this is not a Splunk Supported add-on. You could also try sharing your inputs.conf and some more logs (obfuscated as appropriate) to see if anything else jumps out to the community.

Good Luck!

[ejwade]

Thank you - this is good information on how the TA works. I'm working with Qualys support now and hoping to find an answer. I gave them feedback about their lack of exception handling. 🙂