Splunk Search

How to filter out all events that have duplicate values from my search results?

tnoelOTS
Explorer

I am running a search of my Rapid7 data I need to compare 2 fields Dest_ip and signature_id If both fields have the same data I want to remove all records that have that data from my search.

Example

event 1: dest_ip=10.10.10.10 signature_id=1
event 2: dest_ip=10.10.10.10 signature_id=1
event 3: dest_ip=10.10.10.10 signature_id=2

results after search would only give me the unique value for event 3

0 Karma

sundareshr
Legend

Try the dedup command

base search | dedup dest_ip signature_id

https://docs.splunk.com/Documentation/Splunk/6.5.0/SearchReference/Dedup

UPDATED

base search | eventstats count by dest_ip signature_id | where count=1

tnoelOTS
Explorer

The Dedup command is not working for this application because it returns 1 of the results that had a duplicate value so in my example above dedup gives me Event 1 and event 3 I want to only get event 3 from the search results.

0 Karma

aaraneta_splunk
Splunk Employee
Splunk Employee

@tnoelOTS - Did sundareshr's updated search provide a working solution to your question? If yes, please don't forget to resolve this post by clicking "Accept". If no, please leave a comment with more feedback. Thanks!

0 Karma

sundareshr
Legend

Try the updated search

0 Karma

gokadroid
Motivator

Updated search should work.

0 Karma
Get Updates on the Splunk Community!

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Your Voice Matters! Help Us Shape the New Splunk Lantern Experience

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Building Momentum: Splunk Developer Program at .conf25

At Splunk, developers are at the heart of innovation. That’s why this year at .conf25, we officially launched ...