Getting Data In

duplicate data indexing

shivanandbm
Explorer

i see duplicate data getting indexed.its impacting license. can you please suggest how i can fix this.below is the monitoring .
[monitor:///incoming/XXXXX/XXXX/XXXXX.gz]
disabled = false

whitelist = ..gz

index = XXXXX
sourcetype = XXXX
EVENT_BREAK_ENABLE=true

Tags (1)
0 Karma

MuS
SplunkTrust
SplunkTrust

Please add your outputs.conf to this post.

cheers, MuS

0 Karma

sathwikr076
Communicator

Hi, How many indexers you have and how you are confirming that the duplicate data is ingesting. If those duplicate logs are from same application servers and exactly same in format, Splunk is smart enough to drop the duplicate logs.

0 Karma
Get Updates on the Splunk Community!

Data Management Digest – December 2025

Welcome to the December edition of Data Management Digest! As we continue our journey of data innovation, the ...

Index This | What is broken 80% of the time by February?

December 2025 Edition   Hayyy Splunk Education Enthusiasts and the Eternally Curious!    We’re back with this ...

Unlock Faster Time-to-Value on Edge and Ingest Processor with New SPL2 Pipeline ...

Hello Splunk Community,   We're thrilled to share an exciting update that will help you manage your data more ...