I would suggest that you have Splunk monitor the directory where the files are placed. Splunk will pick up each file as it is written.
Also, for long-term efficiency, I would have a script that regularly removes the old log files from this directory. It probably would not need to run more than once a week.
