How do I dedupe splunk results?

[jtlewis]

Sometimes in splunk I get a lot of duplicate results, is there a dedupe command I can use to narrow the results?

[lucorrea]

I'm having the same problem with dedup.
Has anyone been able to use it without losing all results?
Or maybe you have a different command that can help removing duplicates?
Thanks.

[sankeerthankala]

use the | dedup before the | table pipe, it should work
more like below

Latency_-Session_Average > 50
| search host=servername
| sort -size -Latency-Session_Average
|dedup USERNAME
|table UserName, host, Latency-_Session_Average, RACF_ID, Event_Date, Event_Time, ICA_Name

[mendesjo]

same problem..

[wsmierciak]

I tried this and all of my results disappear and i have 0 results.

My code is:
Latency_-Session_Average > 50
| search host=servername
| sort -size -Latency-Session_Average
|table UserName, host, Latency-_Session_Average, RACF_ID, Event_Date, Event_Time, ICA_Name
|dedup USERNAME

If I remove dedup, i get all results, but multiples of each user in the results

[jsven7]

Same problem

[subash730]

I need all the duplicates also displayed in the table command. But table command only displays unique values for fields. How to display all duplicate values using Table command.

Suppose I have 8 fields to be displayed and two of those fields have unique values for each and every row of data and all other 6 fields have common data, table displays all those 6 fields data once and displays these two fields data only in bulk.

[lpolo · Accepted solution]

You can use the search command dedup. Example:

|dedup name_of_your_field

More information:
http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/dedup