We have been using the Qualys TA to ingest Vulnerability data for quite some time now and notice that occasionally, there is a mis-match in the data seen in Qualys vs the data ingested in Splunk. It eventually catches up (we guess in subsequent runs) but we would prefer it to be available to support reporting requirements.
A few input parameters to quote
- We are on the most recent version of the TA
- We use multi threads
- The TA runs on a HF sending to the indexers
- Internal logs with debug mode enabled seems to indicate no errors whatsoever. The following query indicates that logging has occurred for approximately the same number of hosts everyday. I am deliberately ignoring a few messages to refine the transaction output
index=_internal host=<host> sourcetype=*qualys* host_detection NOT "Will not run" NOT "Running for host_detection" NOT "getting idset inbound queue..." NOT "inboundqueue empty" NOT "waiting for more work"
| transaction startswith="Running Now" endswith="Qualys Host Detection Populator finished."
| convert ctime(_time) AS Time
|table Time,_raw
|rename _raw AS "Job Run History"
The scan completed the previous evening and the Splunk pull happened the next day early hours. Qualys has the data but Splunk did not which doesn't make sense.
A few questions.
- Has anybody else encountered issues and if so, what settings would you recommend to address this?
- Anything obvious that we are missing that is causing the issue in the first place?
- Does the Qualys data get published to be accessed by the API immediately? If it doesn't , we will move our job in-line with that
- Any limits that we should be aware of?
Any inputs are appreciated. Tagging @prabhasgupte as I see you have replied to previous queries on Qualys!
