In my case, it was a misunderstanding of ignoreOlderThan= in inputs.conf.
ignoreOlderThan will completely ignore files that ever reach this threshold.
From the inputs.conf documentation.
"Do NOT select a time that files you want to read could reach in age, even temporarily"
My files wouldn't write to the logs for several weeks and then begin writing again. Splunk would not even try to ingest them.
... View more
I had the same symptoms, it was a configuration issue.
Make sure you fully understand ignoreOlderThan=
https://docs.splunk.com/Documentation/Splunk/6.6.1/Admin/Inputsconf
In my case logs were not written to for 7 + days and then splunk will no longer try to read from that file even when new events appeard in the file..
... View more