Hi,
I see duplicate data getting ingested when a file which was already ingested is being recreated upon a system failure with the existing data plus new data in the file.
For example, lets say test.csv has the following data,
a b c
When the file is deleted and recreated with the same name but with the following additional data,
a b c
1 2 3
4 5 6
it ingests a b c again besides 1 2 3...
Can someone help me with the correct stanza to be added in inputs.conf or any other solution to avoid data being duplicated as per an example mentioned above.
Thanks.
... View more