I currently write status files and collect them into Splunk using monitor statements and Universal Forwarders. Pretty vanilla stuff. Well, we are looking to rewrite our status scripts and I've been asked if we can just rewrite the status information into the same file every 3 minutes and save it. The file size would not just(much), just the contents would change. My first reaction was a faint yet. I thought forwarder would notice a CRC change and reindex from the beginning. This doesn't seem to be happening. If I restart the forwarder is throws and error and reindexes, and it seems once in a while it will catch a change, but not dependably. I assumed placing a timestamp at the head of the status information would assure a CRC change, but sadly we are not getting the desired results.
Ignoring the advisability of this method, should it work? Is there a timeout or polling interval which applies here, at which time the forwarder will recheck CRC?
Log from Forwarder on restart
02-19-2015 09:29:37.599 -0600 INFO WatchedFile - Checksum for seekptr didn't match, will re-read entire file='D:\SplunkData\Test.txt
Here is the contents of my test file. The only required change every 3 minutes is the time stamp, the rest of the text could be identical.