Splunk Search

Retrieve discarded values from the dedup command or other commands?

Splunk Employee
Splunk Employee

I want to use the dedup command and see which values it removes from a field. Is this possible?

Labels (1)
Tags (1)
0 Karma


I suppose you can calculate which ones are removed by dedup, just not by using dedup.

| eventstats last(curious) as remaining by deduping
| stats values(curious) as all by deduping remaining
| eval removed = mvmap(all, if(all == remaining, null(), all))

Here, curious is the field you want to examine, and deduping is the field on which to dedup.

Tags (1)


Use the keepevents option to retain the duplicate events, but without the repeated field value.  Look for where the field is set to null.

If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...