Splunk Search

Alternate to dedup

rupesh26
Path Finder

Hi,

I need to remove duplicates in my results, is there anyway to do this other than using dedup.
I used stats, eventstats still no luck

Tags (2)

nick405060
Motivator

stats count by your_field is faster than dedup if you don't want to keep other fields

rupesh26
Path Finder

Thanks nick, by I want to keep other fields as well to add it to a dashboard.

0 Karma

nick405060
Motivator

Terribly inelegant, but you could stats count by your_field and then join those results with the same search copied and pasted

<your_search> ... | table your_field b c | stats count by your_field | join type=left your_field [<your_search>] | table your_field b c

rupesh26
Path Finder

Really appreciate it Nick , I will try these options.

0 Karma

nick405060
Motivator

Also for reference

https://answers.splunk.com/answers/789749/dedup-vs-stats-performance.html

(I am on the same page as you in that 99.9999% of the time I want to keep my other fields as well, which makes stats values absolutely useless in this "debate")

0 Karma

jscraig2006
Communicator

@ rupesh26 try a distinct count:
| stats dc(<your_feild>)

0 Karma

nick405060
Motivator

This counts distinct values it does not dedup.

0 Karma

rupesh26
Path Finder

Yes, that's correct

0 Karma

nick405060
Motivator

So... this does not answer the question lol

0 Karma

jscraig2006
Communicator

Apologies! I should have read the question more carefully!

0 Karma
Get Updates on the Splunk Community!

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...

Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions

Financial fraud isn't slowing down. If anything, it's getting more sophisticated. Account takeovers, credit ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...