Getting Data In

Show only duplicated fields

kmattern
Builder

I have customers who upload sets of files every day. The upload is done automatically. Sometimes there will be a hitch in the system and one or more of the file set will be uploaded multiple times. The file names all have the term _seq_ followed by a sequence number. So part of the customer events will look like this:

abcdef_seq_1
abcdef_seq_2
abcdef_seq_2
abcdef_seq_3
abcdef_seq_4

I only want to show only the duplicated upload files, in this case abcdef_seq_2. It shouldn't be that hard but I'm busting my head. What am I missing?

Ultimately I need to put this into a data model for a Pivot.

0 Karma
1 Solution

kmattern
Builder

I think I finally figured it out. This search returns only those IIS events that have duplicate cs_uri-query fields.

sourcetype="iis" cs_uri_query="*_seq*"  
| stats first(cs_uri_query) as DupFile, first(cs_username) as Customer, count(cs_uri_query) AS Duplicates by cs_uri_query  
| where Duplicates>1 
| table Customer, DupFile, Duplicates

View solution in original post

kmattern
Builder

I think I finally figured it out. This search returns only those IIS events that have duplicate cs_uri-query fields.

sourcetype="iis" cs_uri_query="*_seq*"  
| stats first(cs_uri_query) as DupFile, first(cs_username) as Customer, count(cs_uri_query) AS Duplicates by cs_uri_query  
| where Duplicates>1 
| table Customer, DupFile, Duplicates

yannK
Splunk Employee
Splunk Employee

ps : please mark your question as answered with the left checkbox to accept your own answer 🙂

0 Karma

yannK
Splunk Employee
Splunk Employee

this is the good method.

to find a dulpicate field
* | stats count by myfield | where count>1

to look at the whole events
* | stats count by _raw | where count>1

somesoni2
Revered Legend

In splunk, do you see duplicate data for the files uploaded multiple times?

0 Karma
Get Updates on the Splunk Community!

Get Operational Insights Quickly with Natural Language on the Splunk Platform

In today’s fast-paced digital world, turning data into actionable insights is essential for success. With ...

What’s New in Splunk Observability Cloud – June 2025

What’s New in Splunk Observability Cloud – June 2025 We are excited to announce the latest enhancements to ...

Almost Too Eventful Assurance: Part 2

Work While You SleepBefore you can rely on any autonomous remediation measures, you need to close the loop ...