Getting Data In

We are on Splunk 7.3.1 , would need to know if there is any to de-duplicate the incoming data on FOrwarder or Indexer level?

rashi83
Explorer

We have many instances where duplicate data has been coming due to server instances running in wrong manner. Because of that duplicate has found to be indexed.
Is there is config change we can do on Forwarder or Indexer to stop indexing supplicate entries.

1 Solution

richgalloway
SplunkTrust
SplunkTrust

Neither forwarder nor indexer can deduplicate events at index time. You must correct the manner in which the server instances run.
You can, however, use dedup to deduplicate events at search time or use the delete command to hide the duplicates.

---
If this reply helps you, an upvote would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Neither forwarder nor indexer can deduplicate events at index time. You must correct the manner in which the server instances run.
You can, however, use dedup to deduplicate events at search time or use the delete command to hide the duplicates.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>