Deduplication at ingest time is tricky bordering on impossible. Duplicate events may follow different paths and so be undetectable until search time. Even if they follow a common path, they may be sufficiently far apart that the EP/IP worker can't detect the duplication.
---
If this reply helps you, Karma would be appreciated.
