topic Re: How to debug why a universal forwarder is reading all log files except one? in Getting Data In

How to debug why a universal forwarder is reading all log files except one?

darthsplunk — Thu, 23 Oct 2014 15:50:14 GMT

Hello,

I have configured inputs.conf on a universal forwarder. The file contains around 20 entries for log files, however one of them is not being read by Splunk.

Within splunkd I can see:

10-23-2014 14:59:17.762 +0100 INFO  TailingProcessor - Parsing configuration stanza: monitor://C:\my\dir\logfile.log.

I would then expect to see:

TailingProcessor - Adding watch on path for file <…>

and

WatchedFile - Will begin reading at offset=61873 for file <…>

But I never see this. I have performed the following debug steps:

Run "splunk cmd btool inputs list --debug". This shows the stanza in question.
Checked "https://localhost:8089/services/admin/inputstatus/TailingProcessor:FileStatus" which does not show any errors of the input being skipped.

How can I debug this further? The universalforwarder is sending data from other log files on this host ok so it isn't a connection issue.

Any help is appreciated.

Thanks,
DS

Re: How to debug why a universal forwarder is reading all log files except one?

esix_splunk — Thu, 23 Oct 2014 16:29:56 GMT

Check file level permissions and make sure the user splunk is running as can read/execute on those files.

Additionally, is this a distributed environment? If so, what does your outputs.conf look like?

Re: How to debug why a universal forwarder is reading all log files except one?

the_wolverine — Thu, 23 Oct 2014 21:56:50 GMT

You should search the _internal index for any reference to the log file name in question. Some issues I have seen in the field are: thruput set to low (forwarder is not able to consume the log before it rolls), permissions issues (splunk doesn't have read access to the log file.)

What is the version of Splunk forwarder in question?

Re: How to debug why a universal forwarder is reading all log files except one?

jrodman — Fri, 24 Oct 2014 05:07:55 GMT

Adding watch on path is for the common parent of the monitored locations. Do you have monitors on directories that are above that location, such as c:\my or c:\my\dir? What is the set of directories that you see for these lines? What is the set of monitor stanzas that you have?

Do you have any messages relating to logfile.log in splunkd.log at all?

Re: How to debug why a universal forwarder is reading all log files except one?

darthsplunk — Tue, 24 Feb 2015 16:20:13 GMT

It turned out to be an issue with other entries in the inputs.conf where wildcards were in use. These were corrected and the inputs work as expected. Thanks all.

Re: How to debug why a universal forwarder is reading all log files except one?

dailv1808 — Wed, 26 Jul 2017 09:26:50 GMT

I have same problem. Plz let's show me how do you fix it. thanks