topic Re: Monitoring log files reused in a ring? in Splunk Dev

Monitoring log files reused in a ring?

sowings — Wed, 31 Oct 2012 18:15:46 GMT

I have a log source which re-uses the same log file based upon the day of the month. A filename contains other information, but that information doesn't change. Here is a sample: 10_40809_(IRL_Lab_Live).log. That filename represents log data written on the 10th of October. Next month, it will be truncated, starting again from an empty file, but still the same filename. I've got a standard [monitor:...] style inputs.conf stanza monitoring the directory containing these files.

It seems that I have to occasionally jiggle Splunk's elbow (restart) to get it to index "today's" data. Is there some other trick to get Splunk to index this smoothly? It doesn't sound like a candidate for alwaysOpenFile, but I could try that setting if it would help.

Re: Monitoring log files reused in a ring?

ShaneNewman — Fri, 22 Mar 2013 21:50:54 GMT

I discovered that if we removed the ignoreOlderThan filter on the inputs and just ate the bullet on indexing all of that data that it will work fine. For whatever reason I guess that the filter was looking at the original file time?

Re: Monitoring log files reused in a ring?

sowings — Fri, 22 Mar 2013 21:52:11 GMT

Possible. Windows log sources are ... notorious for not updating the modtime of the file. It's seen a lot in IIS installations.