Splunk Search

How to filter out all events that have duplicate values from my search results?

tnoelOTS
Explorer

I am running a search of my Rapid7 data I need to compare 2 fields Dest_ip and signature_id If both fields have the same data I want to remove all records that have that data from my search.

Example

event 1: dest_ip=10.10.10.10 signature_id=1
event 2: dest_ip=10.10.10.10 signature_id=1
event 3: dest_ip=10.10.10.10 signature_id=2

results after search would only give me the unique value for event 3

0 Karma

sundareshr
Legend

Try the dedup command

base search | dedup dest_ip signature_id

https://docs.splunk.com/Documentation/Splunk/6.5.0/SearchReference/Dedup

UPDATED

base search | eventstats count by dest_ip signature_id | where count=1

tnoelOTS
Explorer

The Dedup command is not working for this application because it returns 1 of the results that had a duplicate value so in my example above dedup gives me Event 1 and event 3 I want to only get event 3 from the search results.

0 Karma

aaraneta_splunk
Splunk Employee
Splunk Employee

@tnoelOTS - Did sundareshr's updated search provide a working solution to your question? If yes, please don't forget to resolve this post by clicking "Accept". If no, please leave a comment with more feedback. Thanks!

0 Karma

sundareshr
Legend

Try the updated search

0 Karma

gokadroid
Motivator

Updated search should work.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...