Getting Data In

How to count the number of duplicates detected?

nickhaj
New Member

Hi,

I want to know how many duplicates of a filename (in field Target_file) have been detected for events indexed daily (for 2 incoming files, 1 outgoing - the field Target_file is common to all 3 file transfers); I am expecting the result to be zero 99.99% of the time as the filenames should always be unique; the result then being displayed on a dashboard panel where it will go red if the count is greater than zero.

I've tried faffing with dedup but that seems to count the unique occurences (3 per day) rather than duplicates detected so does not give my anticipated result (0)

I've looked thru loads of the suggestions but can't seem to find this exact scenario, but I am a Splunk Beginner so BIG apols if I have missed it somehwere 🙂

Many Thanks for any assistance!

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Not sure if this was in the load of suggestions you've already tried.

... | stats count by Target_file | where count > 1 | ...
---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

nickhaj
New Member

Hi Rich - got there I think ....

After finding the target records I add your suggested search and then

| table Target_File

to the end of it....

This returns 'no results found' into the Dashboard Panel if, as expected, no duplicates are detected.......and the Target_file value should a duplicate be detected.

So your suggestion was spot on, I just needed to suss the end bit.

Many Thanks for your guidance!

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Not sure if this was in the load of suggestions you've already tried.

... | stats count by Target_file | where count > 1 | ...
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

What’s new on Splunk Lantern in August

This month’s Splunk Lantern update gives you the low-down on all of the articles we’ve published over the past ...

Welcome to the Future of Data Search & Exploration

You have more data coming at you than ever before. Over the next five years, the total amount of digital data ...

This Week's Community Digest - Splunk Community Happenings [8.3.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...