Splunk Search

Alternate to dedup

rupesh26
Path Finder

Hi,

I need to remove duplicates in my results, is there anyway to do this other than using dedup.
I used stats, eventstats still no luck

Tags (2)

nick405060
Motivator

stats count by your_field is faster than dedup if you don't want to keep other fields

rupesh26
Path Finder

Thanks nick, by I want to keep other fields as well to add it to a dashboard.

0 Karma

nick405060
Motivator

Terribly inelegant, but you could stats count by your_field and then join those results with the same search copied and pasted

<your_search> ... | table your_field b c | stats count by your_field | join type=left your_field [<your_search>] | table your_field b c

rupesh26
Path Finder

Really appreciate it Nick , I will try these options.

0 Karma

nick405060
Motivator

Also for reference

https://answers.splunk.com/answers/789749/dedup-vs-stats-performance.html

(I am on the same page as you in that 99.9999% of the time I want to keep my other fields as well, which makes stats values absolutely useless in this "debate")

0 Karma

jscraig2006
Communicator

@ rupesh26 try a distinct count:
| stats dc(<your_feild>)

0 Karma

nick405060
Motivator

This counts distinct values it does not dedup.

0 Karma

rupesh26
Path Finder

Yes, that's correct

0 Karma

nick405060
Motivator

So... this does not answer the question lol

0 Karma

jscraig2006
Communicator

Apologies! I should have read the question more carefully!

0 Karma
Get Updates on the Splunk Community!

New Cloud Intrusion Detection System Add-on for Splunk

In July 2022 Splunk released the Cloud IDS add-on which expanded Splunk capabilities in security and data ...

Happy CX Day to our Community Superheroes!

Happy 10th Birthday CX Day!What is CX Day? It’s a global celebration recognizing innovation and success in the ...

Check out This Month’s Brand new Splunk Lantern Articles

Splunk Lantern is a customer success center providing advice from Splunk experts on valuable data insights, ...