dedup and unique difference

[logloganathan]

Could you please explain the difference between dedup and unique

[woodcock]

The uniq command removes any search result if that result is an exact duplicate so the events must be resorted to use it. I have NEVER had any occasion to use this command. Ever. The dedup command is MUCH more flexible. Unlike uniq It can be map-reduced, it can trim to a certain size (defaults to 1) and can apply to any number of fields at the same time.

[bmunson_splunk · Accepted solution]

The uniq command removes duplicates if the whole event or row of a table are the same. It takes no fields or options as everything is checked. It is an ideal command if you have duplicate data.
See docs on uniq for more detail.

The dedup command looks only at the fields you tell it to. So if I say "| dedup host", it only looks at the host field and keeps the first from each host. You can specify multiple fields and has options like consecutive (only remove events with duplicate combinations of values that are in consecutive rows.) or keepempty (also keep events that do not have the requested field).
See docs on dedup for more detail

[niketn]

Agree, please use Splunk Documentation as your first point of research, or be more specific which what is your use case or reason for the question.

[niketn]

@logloganathan, I see that you have down voted my comment. Down voting should only be reserved for suggestions/solutions that could be potentially harmful for a Splunk environment or goes completely against known best practices.

Simply commenting with more information about what didn't work and what you've tried (or whatever other info may be relevant) would suffice to help you troubleshoot further.

Refer to community guidelines (ironically again on Splunk Docs :)): https://docs.splunk.com/Documentation/Splunkbase/splunkbase/Answers/Splunkcommunityguidelines

I am curious to know as to how request to research on own before asking question is harmful for you/your environment. Please clarify!!!