Splunk Search

Alternate to dedup

rupesh26
Path Finder

Hi,

I need to remove duplicates in my results, is there anyway to do this other than using dedup.
I used stats, eventstats still no luck

Tags (2)

nick405060
Motivator

stats count by your_field is faster than dedup if you don't want to keep other fields

rupesh26
Path Finder

Thanks nick, by I want to keep other fields as well to add it to a dashboard.

0 Karma

nick405060
Motivator

Terribly inelegant, but you could stats count by your_field and then join those results with the same search copied and pasted

<your_search> ... | table your_field b c | stats count by your_field | join type=left your_field [<your_search>] | table your_field b c

rupesh26
Path Finder

Really appreciate it Nick , I will try these options.

0 Karma

nick405060
Motivator

Also for reference

https://answers.splunk.com/answers/789749/dedup-vs-stats-performance.html

(I am on the same page as you in that 99.9999% of the time I want to keep my other fields as well, which makes stats values absolutely useless in this "debate")

0 Karma

jscraig2006
Communicator

@ rupesh26 try a distinct count:
| stats dc(<your_feild>)

0 Karma

nick405060
Motivator

This counts distinct values it does not dedup.

0 Karma

rupesh26
Path Finder

Yes, that's correct

0 Karma

nick405060
Motivator

So... this does not answer the question lol

0 Karma

jscraig2006
Communicator

Apologies! I should have read the question more carefully!

0 Karma
Get Updates on the Splunk Community!

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to Officially Supported Splunk ...