https://community.splunk.com/t5/Splunk-Enterprise-Security/datamodel-dedup-count/m-p/381085#M3864 <P>This in regards to vulnerability center from Qualys</P> <P>issue - the datamodel gets updated every 24hrs (this cant change) and when we click in the vulnerability centre we get incorrect numbers. It seems like its counting the number of same vulnerability for the host multiple times which increases the number in the dashboard. I can search the index and get the correct result but the engineer wants to use the defined dashboard in ES.</P> <P>goal - to somehow dedup the count in that datamodel and get the correct asnwer using the datamodel</P> <P>datamodal query that gives incorrect info - <BR /> | tstats <CODE>summariesonly</CODE> count from datamodel=Vulnerabilities.Vulnerabilities where earliest=-30d@d latest=+0s <CODE>cim_filter_vuln_severity("Vulnerabilities")</CODE> by Vulnerabilities.signature,Vulnerabilities.dest </P> <P>eg answer: count should be 1 instead of 8<BR /> Vulnerabilities.signature Vulnerabilities.dest count<BR /> 'nlockmgr' Allows Proxying of NFS Requests 172.20.204.14 8</P> <P>searching index that gives correct answer - <BR /> eventtype="qualys_vm_detection_event" STATUS="NEW" OR STATUS="ACTIVE" earliest=-30d@d latest=+0s | dedup QID |stats count by dest_ip signature</P> <P>eg correct answer<BR /> dest_ip signature count<BR /> 172.20.204.18 'nlockmgr' Allows Proxying of NFS Requests 1</P> Tue, 29 Sep 2020 23:52:50 GMT siddh01r 2020-09-29T23:52:50Z https://community.splunk.com/t5/Splunk-Enterprise-Security/datamodel-dedup-count/m-p/381085#M3864 <P>This in regards to vulnerability center from Qualys</P> <P>issue - the datamodel gets updated every 24hrs (this cant change) and when we click in the vulnerability centre we get incorrect numbers. It seems like its counting the number of same vulnerability for the host multiple times which increases the number in the dashboard. I can search the index and get the correct result but the engineer wants to use the defined dashboard in ES.</P> <P>goal - to somehow dedup the count in that datamodel and get the correct asnwer using the datamodel</P> <P>datamodal query that gives incorrect info - <BR /> | tstats <CODE>summariesonly</CODE> count from datamodel=Vulnerabilities.Vulnerabilities where earliest=-30d@d latest=+0s <CODE>cim_filter_vuln_severity("Vulnerabilities")</CODE> by Vulnerabilities.signature,Vulnerabilities.dest </P> <P>eg answer: count should be 1 instead of 8<BR /> Vulnerabilities.signature Vulnerabilities.dest count<BR /> 'nlockmgr' Allows Proxying of NFS Requests 172.20.204.14 8</P> <P>searching index that gives correct answer - <BR /> eventtype="qualys_vm_detection_event" STATUS="NEW" OR STATUS="ACTIVE" earliest=-30d@d latest=+0s | dedup QID |stats count by dest_ip signature</P> <P>eg correct answer<BR /> dest_ip signature count<BR /> 172.20.204.18 'nlockmgr' Allows Proxying of NFS Requests 1</P> Tue, 29 Sep 2020 23:52:50 GMT https://community.splunk.com/t5/Splunk-Enterprise-Security/datamodel-dedup-count/m-p/381085#M3864 siddh01r 2020-09-29T23:52:50Z https://community.splunk.com/t5/Splunk-Enterprise-Security/datamodel-dedup-count/m-p/381086#M3865 <P>meh.. I kinda disagree.<BR /> Enterprise Security is tracking your security posture across time. If you do a weekly scan and you fixed a load of vulns last week you want to see that number decrease, likewise if you find a load more issues this week ES wants to know that too.</P> <P>I know this sounds a bit counter intuitive, but ES is tracking total vulns 'DETECTED' not unique vulns that exist. Its a subtlety which is related to how often you scan your hosts, but deduping this is not what ES or the correlation searches or notable events expects.</P> <P>If you want to see total unique vulns per host (which is what you appear to be distilling this to) then you should create your own report/dashboard.</P> Wed, 27 Mar 2019 10:47:47 GMT https://community.splunk.com/t5/Splunk-Enterprise-Security/datamodel-dedup-count/m-p/381086#M3865 nickhills 2019-03-27T10:47:47Z