Getting Data In

How do I dedupe splunk results?

jtlewis
Engager

Sometimes in splunk I get a lot of duplicate results, is there a dedupe command I can use to narrow the results?

Tags (2)
1 Solution

lpolo
Motivator

You can use the search command dedup. Example:

|dedup name_of_your_field

More information:
http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/dedup

View solution in original post

lucorrea
New Member

I'm having the same problem with dedup.
Has anyone been able to use it without losing all results?
Or maybe you have a different command that can help removing duplicates?
Thanks.

0 Karma

sankeerthankala
New Member

use the | dedup before the | table pipe, it should work
more like below

Latency_-Session_Average > 50
| search host=servername
| sort -size -Latency-Session_Average
|dedup USERNAME
|table UserName, host, Latency-_Session_Average, RACF_ID, Event_Date, Event_Time, ICA_Name

0 Karma

mendesjo
Explorer

same problem..

0 Karma

wsmierciak
Engager

I tried this and all of my results disappear and i have 0 results.

My code is:
Latency_-Session_Average > 50
| search host=servername
| sort -size -Latency
-Session_Average
|table UserName, host, Latency
-_Session_Average, RACF_ID, Event_Date, Event_Time, ICA_Name
|dedup USERNAME

If I remove dedup, i get all results, but multiples of each user in the results

0 Karma

jsven7
Communicator

Same problem

0 Karma

subash730
New Member

I need all the duplicates also displayed in the table command. But table command only displays unique values for fields. How to display all duplicate values using Table command.

Suppose I have 8 fields to be displayed and two of those fields have unique values for each and every row of data and all other 6 fields have common data, table displays all those 6 fields data once and displays these two fields data only in bulk.

0 Karma

lpolo
Motivator

You can use the search command dedup. Example:

|dedup name_of_your_field

More information:
http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/dedup

View solution in original post

Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!