Monitoring folder stops monitoring files

[keycoldstorage]

I suspect that this has something to do with the fact that my log files are being generated by appending to the end of a flat file.

A monitored folder with two flat files that are being written to is not adding to the index. When I add a test line at the top of the file, Splunk catches that on one file (about 80mb), but not the other (about 3mb). However, it still does not index the additions to the tails of the files.

Do I need to configure tailing? I was under the impression that the folder monitor was supposed to index changes in existing logfiles within the monitored folder.

I should add that these files are written to more than once per five seconds generally. Might that have something to do with my problem? I found this piece of information in the troubleshooter:

Splunk keeps only so many files open at a time (default, 32). If you have files that are written to more than once every 5 seconds, this table should be expandedshould be expanded

Additional information: it appears as though this may have to do with buckets? I have 9 overlapping hot buckets, all of which failing to start splunk-optimize. The errors seem to correspond roughly with the last indexed data in the two logs.

arri

[keycoldstorage]

Just in case some other noob like myself is out there and wonders why this sort of thing might happen, check to see if you've got forwarding enabled. I had turned it on to experiment with it, but didn't realize that, despite checking the store a local copy box, the forwarder would no longer index the data. I then proceeded to ignore the receiver, and forget that I had enabled forwarding, and wonder why it wasn't working right when I came back to it after a month.

Anyway, deleted the forwarding configuration, restarted, and all is well.