https://community.splunk.com/t5/Getting-Data-In/How-to-avoid-duplicated-event/m-p/75495#M15390 <P>We have csv type of data file which is overwritten and with new data appended to the end every night. I found Splunk load/duplicate all the data again everyday! </P> <P>As I know crcSalt only check CRC with first few lines of the file. How Splunk works in this case to identify only end of the file has new data?</P> <P>followTail works for file replace?</P> Wed, 06 Apr 2011 14:59:01 GMT hochit 2011-04-06T14:59:01Z https://community.splunk.com/t5/Getting-Data-In/How-to-avoid-duplicated-event/m-p/75495#M15390 <P>We have csv type of data file which is overwritten and with new data appended to the end every night. I found Splunk load/duplicate all the data again everyday! </P> <P>As I know crcSalt only check CRC with first few lines of the file. How Splunk works in this case to identify only end of the file has new data?</P> <P>followTail works for file replace?</P> Wed, 06 Apr 2011 14:59:01 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-avoid-duplicated-event/m-p/75495#M15390 hochit 2011-04-06T14:59:01Z https://community.splunk.com/t5/Getting-Data-In/How-to-avoid-duplicated-event/m-p/75496#M15391 <P>I am having the exact same issue! did you figure out a solution?</P> Thu, 14 Feb 2013 15:01:01 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-avoid-duplicated-event/m-p/75496#M15391 dlovett 2013-02-14T15:01:01Z https://community.splunk.com/t5/Getting-Data-In/How-to-avoid-duplicated-event/m-p/75497#M15392 <P>How are you loading the file:</P> <P>Are you using a Splunk monitor?<BR /> <A href="http://docs.splunk.com/Documentation/Splunk/5.0.1/Data/Monitorfilesanddirectories">http://docs.splunk.com/Documentation/Splunk/5.0.1/Data/Monitorfilesanddirectories</A></P> <P>That might be your problem.</P> Thu, 14 Feb 2013 19:42:55 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-avoid-duplicated-event/m-p/75497#M15392 barne_dn 2013-02-14T19:42:55Z