How to avoid duplicated event

[hochit]

We have csv type of data file which is overwritten and with new data appended to the end every night. I found Splunk load/duplicate all the data again everyday!

As I know crcSalt only check CRC with first few lines of the file. How Splunk works in this case to identify only end of the file has new data?

followTail works for file replace?

[barne_dn]

How are you loading the file:

Are you using a Splunk monitor?
http://docs.splunk.com/Documentation/Splunk/5.0.1/Data/Monitorfilesanddirectories

That might be your problem.

[dlovett]

I am having the exact same issue! did you figure out a solution?