I have a monitored file input for a .tsv file that gets updated via a SQL query every hour. However, the data is only showing up in the index periodically (haven't been able to determine the frequency, but it isn't hourly like it should be). If I restart the forwarder I see the TailingProcessor add a watch, but the file subsequently gets handled by the BatchReader as shown in the log snippet below. For other files inputs using the [monitor://...] stanza I don't see any log entries related to the BatchReader, any ideas why this one is being treated any differently? Universal forwarder is version 6.2.1.
# grep metrics5.tsv /opt/splunkforwarder/var/log/splunk/splunkd.log
04-27-2015 09:46:13.653 -0400 INFO TailingProcessor - Parsing configuration stanza: monitor:///data/log/hadoop_job_metrics/metrics5.tsv.
04-27-2015 09:46:13.653 -0400 INFO TailingProcessor - Adding watch on path: /data/log/hadoop_job_metrics/metrics5.tsv.
04-27-2015 09:46:13.660 -0400 INFO BatchReader - Removed from queue file='/data/log/hadoop_job_metrics/metrics5.tsv'.
04-27-2015 10:01:47.734 -0400 INFO BatchReader - Removed from queue file='/data/log/hadoop_job_metrics/metrics5.tsv'.