topic Splunk Find Duplicates Within KV Store And Compare Source in All Apps and Add-ons

Splunk Find Duplicates Within KV Store And Compare Source

joemarty82 — Mon, 31 Aug 2020 17:58:58 GMT

Hello,

I am having problems trying to find duplicate entries within my splunk kvstore. Basically, what I want to do is find duplicates based on a few fields such as FQDN, CVE, and PORT. Then, once I found duplicates I just want to output them in a table if their SOURCE field is different. The query I have so far is:

| inputlookup vul_kvstore | stats count by fqdn, port, cve | where count>1 | table fqdn, port, cve, source

The problem I have now is in my table I do not have access to the source field as it looks like the stats count line basically pulls out only the fqdn, port, and cve data. How do I get access to the source field data? Maybe I just have to revise my original query so I do not loose data to that field but so far nothing I try works. Hopefully someone can provide me some advise to push me through this problem.

Thanks,

Joe

Re: Splunk Find Duplicates Within KV Store And Compare Source

thambisetty — Mon, 31 Aug 2020 18:48:24 GMT

| inputlookup vul_kvstore

| stats dc(source) as distinct_source_count values(source) as source by fqdn, port, cve

| where distinct_source_count > 1

| table fqdn, port, cve, source

Re: Splunk Find Duplicates Within KV Store And Compare Source

joemarty82 — Mon, 31 Aug 2020 19:48:01 GMT

Hey Tham,

Thanks for the input. However, this will not work because the source field only contains one entry in each kvstore record. Looks like your solution assumes there are more than 1.

Re: Splunk Find Duplicates Within KV Store And Compare Source

Nisha18789 — Tue, 01 Sep 2020 08:38:49 GMT

Hi @joemarty82 , can you try this

| inputlookup vul_kvstore | stats count by source fqdn, port, cve | where count>1 | table source, fqdn, port, cve