topic Re: duplicate data indexing in Getting Data In

duplicate data indexing

shivanandbm — Wed, 30 Sep 2020 01:54:02 GMT

i see duplicate data getting indexed.its impacting license. can you please suggest how i can fix this.below is the monitoring .
[monitor:///incoming/XXXXX/XXXX/XXXXX.gz]
disabled = false

whitelist = ..gz

index = XXXXX
sourcetype = XXXX
EVENT_BREAK_ENABLE=true

Re: duplicate data indexing

sathwikr076 — Mon, 26 Aug 2019 20:29:54 GMT

Hi, How many indexers you have and how you are confirming that the duplicate data is ingesting. If those duplicate logs are from same application servers and exactly same in format, Splunk is smart enough to drop the duplicate logs.

Re: duplicate data indexing

MuS — Mon, 26 Aug 2019 20:32:45 GMT

Please add your outputs.conf to this post.

cheers, MuS