We are monitoring many files with the UF using the [monitor] stanza. For housekeeping reasons we need to delete the files every now and then. Now deleting a file that was not already fully transferred to the indexer would lead to data loss.
How can we find out which files have already been transferred by the UF?
Of course the UF itself does not know that a file is already complete or closed. It only sees that there was not more data added since some time. So what we need is some kind of list that shows the filenames being monitored as well as the file pointer up to what amount was already transferred.
example output:
/var/adm/messages 1233122
/var/adm/authlog 1233
Based on that our cleanup mechanism could then compare the actual size of a closed file with the size reported in the output above. If they are equal it knows that the UF has transferred everything and we can delete the file.
An alternative would also be to provide an "auto-close" function in the UF. This function would trigger the execution of a shell command (specified in inputs.conf) as soon as the file's size has not changed for "auto-close-seconds" (also specified in intputs.conf).
