Splunk Search

How to filter out all events that have duplicate values from my search results?

tnoelOTS
Explorer

I am running a search of my Rapid7 data I need to compare 2 fields Dest_ip and signature_id If both fields have the same data I want to remove all records that have that data from my search.

Example

event 1: dest_ip=10.10.10.10 signature_id=1
event 2: dest_ip=10.10.10.10 signature_id=1
event 3: dest_ip=10.10.10.10 signature_id=2

results after search would only give me the unique value for event 3

0 Karma

sundareshr
Legend

Try the dedup command

base search | dedup dest_ip signature_id

https://docs.splunk.com/Documentation/Splunk/6.5.0/SearchReference/Dedup

UPDATED

base search | eventstats count by dest_ip signature_id | where count=1

tnoelOTS
Explorer

The Dedup command is not working for this application because it returns 1 of the results that had a duplicate value so in my example above dedup gives me Event 1 and event 3 I want to only get event 3 from the search results.

0 Karma

aaraneta_splunk
Splunk Employee
Splunk Employee

@tnoelOTS - Did sundareshr's updated search provide a working solution to your question? If yes, please don't forget to resolve this post by clicking "Accept". If no, please leave a comment with more feedback. Thanks!

0 Karma

sundareshr
Legend

Try the updated search

0 Karma

gokadroid
Motivator

Updated search should work.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...