Getting Data In

How to count the number of duplicates detected?

nickhaj
New Member

Hi,

I want to know how many duplicates of a filename (in field Target_file) have been detected for events indexed daily (for 2 incoming files, 1 outgoing - the field Target_file is common to all 3 file transfers); I am expecting the result to be zero 99.99% of the time as the filenames should always be unique; the result then being displayed on a dashboard panel where it will go red if the count is greater than zero.

I've tried faffing with dedup but that seems to count the unique occurences (3 per day) rather than duplicates detected so does not give my anticipated result (0)

I've looked thru loads of the suggestions but can't seem to find this exact scenario, but I am a Splunk Beginner so BIG apols if I have missed it somehwere 🙂

Many Thanks for any assistance!

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Not sure if this was in the load of suggestions you've already tried.

... | stats count by Target_file | where count > 1 | ...
---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

nickhaj
New Member

Hi Rich - got there I think ....

After finding the target records I add your suggested search and then

| table Target_File

to the end of it....

This returns 'no results found' into the Dashboard Panel if, as expected, no duplicates are detected.......and the Target_file value should a duplicate be detected.

So your suggestion was spot on, I just needed to suss the end bit.

Many Thanks for your guidance!

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Not sure if this was in the load of suggestions you've already tried.

... | stats count by Target_file | where count > 1 | ...
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...