Are you a member of the Splunk Community?
- Find Answers
- :
- Using Splunk
- :
- Splunk Search
- :
- Re: Alternate to dedup
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Alternate to dedup
Hi,
I need to remove duplicates in my results, is there anyway to do this other than using dedup.
I used stats, eventstats still no luck
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
stats count by your_field is faster than dedup if you don't want to keep other fields
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks nick, by I want to keep other fields as well to add it to a dashboard.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Terribly inelegant, but you could stats count by your_field and then join those results with the same search copied and pasted
<your_search> ... | table your_field b c | stats count by your_field | join type=left your_field [<your_search>] | table your_field b c
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Really appreciate it Nick , I will try these options.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Also for reference
https://answers.splunk.com/answers/789749/dedup-vs-stats-performance.html
(I am on the same page as you in that 99.9999% of the time I want to keep my other fields as well, which makes stats values absolutely useless in this "debate")
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@ rupesh26 try a distinct count:
| stats dc(<your_feild>)
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This counts distinct values it does not dedup.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, that's correct
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So... this does not answer the question lol
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Apologies! I should have read the question more carefully!
Tech Talk Recap | Mastering Threat Hunting
Observability for AI Applications: Troubleshooting Latency
Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?
