This answer is cloned from another Q/A here: https://answers.splunk.com/answers/309910/how-to-monitor-a-folder-for-newest-files-only-file.html You might think to try ingnoreOlderThan but if you do, beware that it does not work the way most people think that it does: once Splunk ignores the file the first time, it is in a blacklist and it will never be examined again, even if new data goes into it! It is the opposite of what you need anyway. Here is an interesting read on that feature: http://answers.splunk.com/answers/242194/missing-events-from-monitored-logs.html Also read here, too: http://answers.splunk.com/answers/57819/when-is-it-appropriate-to-set-followtail-to-true.html I have used the following hack to solve this problem: Create a new directory somewhere else (/destination/path/) and point the Splunk forwarder there. Then setup a cron job that creates selective soft links to files in the real directory (/source/path/) for any file that has been touched in the last 5 minutes (or whatever your threshold is), like this: */5 * * * * cd /source/file/path/ && /bin/find . -maxdepth 1 -type f -mmin -5 | /bin/sed "s/^..//" | /usr/bin/xargs -I {} /bin/ln -fs /source/path/{} /destination/path/{} The nice thing about this hack is that you can create a similar cron job to remove files that have not been changed in a while (because if you have too many files to sort through, even if they have no new data, your forwarder will slow WAY down) and if they ever do get touched, the first cron will add them back! Don't forget to setup a 2nd cron to delete the softlinks, too, with whatever logic allows you to be sure that the file will never be used again, or you will end up with tens of thousands of files here, too. ... View more