topic Re: Retrieve discarded values from the dedup command or other commands? in Splunk Search

Retrieve discarded values from the dedup command or other commands?

amorales_splunk — Fri, 13 Jan 2023 20:14:20 GMT

I want to use the dedup command and see which values it removes from a field. Is this possible?

Re: Retrieve discarded values from the dedup command or other commands?

richgalloway — Fri, 13 Jan 2023 21:28:08 GMT

Use the keepevents option to retain the duplicate events, but without the repeated field value. Look for where the field is set to null.

Re: Retrieve discarded values from the dedup command or other commands?

yuanliu — Fri, 13 Jan 2023 21:32:49 GMT

I suppose you can calculate which ones are removed by dedup, just not by using dedup.

| eventstats last(curious) as remaining by deduping | stats values(curious) as all by deduping remaining | eval removed = mvmap(all, if(all == remaining, null(), all))

Here, curious is the field you want to examine, and deduping is the field on which to dedup.