My splunk instance monitored the directory where proxy server upload compressed access log to via ftp. However my splunk instance sometimes indexed event twice, it result in duplicate events.
I got answer by splunk engineer that Splunk tries hard to read uncompleted file. Sometimes it fails to read. Other time splunk might be able to read. Upload the file to one directory where Splunk is not monitoring, and move it to the directory splunk is monitoring.
My question is if anybody have experience to move the uploaded file to directory splunk is monitoring, please share your experience with me. I think splunk can not do it, I will need to achieve it by using os command or script. I would like to know which os command or what script you used and move the file safely.
