datamodel - dedup count

[siddh01r]

This in regards to vulnerability center from Qualys

issue - the datamodel gets updated every 24hrs (this cant change) and when we click in the vulnerability centre we get incorrect numbers. It seems like its counting the number of same vulnerability for the host multiple times which increases the number in the dashboard. I can search the index and get the correct result but the engineer wants to use the defined dashboard in ES.

goal - to somehow dedup the count in that datamodel and get the correct asnwer using the datamodel

datamodal query that gives incorrect info -
| tstats summariesonly count from datamodel=Vulnerabilities.Vulnerabilities where earliest=-30d@d latest=+0s cim_filter_vuln_severity("Vulnerabilities") by Vulnerabilities.signature,Vulnerabilities.dest

eg answer: count should be 1 instead of 8
Vulnerabilities.signature Vulnerabilities.dest count
'nlockmgr' Allows Proxying of NFS Requests 172.20.204.14 8

searching index that gives correct answer -
eventtype="qualys_vm_detection_event" STATUS="NEW" OR STATUS="ACTIVE" earliest=-30d@d latest=+0s | dedup QID |stats count by dest_ip signature

eg correct answer
dest_ip signature count
172.20.204.18 'nlockmgr' Allows Proxying of NFS Requests 1

[nickhills]

meh.. I kinda disagree.
Enterprise Security is tracking your security posture across time. If you do a weekly scan and you fixed a load of vulns last week you want to see that number decrease, likewise if you find a load more issues this week ES wants to know that too.

I know this sounds a bit counter intuitive, but ES is tracking total vulns 'DETECTED' not unique vulns that exist. Its a subtlety which is related to how often you scan your hosts, but deduping this is not what ES or the correlation searches or notable events expects.

If you want to see total unique vulns per host (which is what you appear to be distilling this to) then you should create your own report/dashboard.