Hello,
We have a use case.
Using the Splunk DB Connect, we ingest data from the various systems especially from the ERP.
Every change on an article in the ERP is pushed into a temp DB which is monitored by the SPLUNK DB connect.
There a millions of data movements each day.
But in the end of the day, we just need to work with the latest unique data that are in the system for each article. Each event has some 10-30 fields.
What is the best way to getting rid of all the duplicates that are comming into the system ? Delete ? How ? skip ? Lookup ? Summary DB ? How ?
What are the ideas that you might have or maybe some ideas i'm missing ?
... View more