topic Re: Splunk Forwarder Tail Fails in Getting Data In

Splunk Forwarder Tail Fails

OldManEd — Thu, 26 Jun 2014 16:16:08 GMT

I have several forwarders that are release 4.3.2. The issue is that the log files they are configured to send to my indexers periodically rotate. For some reason Splunk will not send any more data when this happens. But, if I restart the forwarder, data starts coming in with no problems. The strange thing is that this only happens on some of the forwarders.

File permissions were checked and found to be OK.

The related errors in the splunkd.log file are;

06-14-2014 08:00:08.608 -0600 ERROR TailingProcessor - Ignoring path due to: failed to open for checksum: '/var/tmp/xxx/xxx.log' (No such file or directory)

06-14-2014 08:00:09.911 -0600 ERROR TailingProcessor - Unable to resolve path for symlink: /var/tmp/xxx/xxx.log.

Any ideas?

Re: Splunk Forwarder Tail Fails

abonuccelli_spl — Thu, 26 Jun 2014 17:06:52 GMT

06-14-2014 08:00:09.911 -0600 ERROR
TailingProcessor - Unable to resolve
path for symlink:
/var/tmp/xxx/xxx.log.

can you actually resolve that symlink using ls?

tail /path/to/symlink

Re: Splunk Forwarder Tail Fails

OldManEd — Mon, 28 Sep 2020 16:56:24 GMT

Yes. The link is "/var/tmp/xxx/xxx.log -> /usr/xxx/yyy/AAA/work/xxx_aaa_99.log". From what I understand, the target of the link, in this case "/usr/xxx/yyy/AAA/work/xxx_aaa_99.log", is somehow "processed" and there is a period of about 1.0 to 1.5 seconds where it doesn't exist. But once the process is complete, everything functions correctly except the forwarder doesn't continue to tail the new file.

Re: Splunk Forwarder Tail Fails

the_wolverine — Thu, 26 Jun 2014 17:39:11 GMT

Splunk bugs go unreported (or undetected) often. I suggest upgrading to a later version of the forwarder, at least version 4.3.4 or 5.0.7 to see if you still encounter the issue.

Re: Splunk Forwarder Tail Fails

scc00 — Tue, 11 Aug 2015 15:51:09 GMT

So I had the same issue. The fix was upping the permissions to the symlink. So check to see if the user for your splunk install can get to the full location and increase the permissions accordingly.

Re: Splunk Forwarder Tail Fails

kamaljagga — Fri, 09 Aug 2019 17:42:25 GMT

I am facing the same issue and my forwarder version is 4.2.6 and forwarder runs as root. Thinking of doing a clean re-install of forwarder.
If anyone found any other solution. Kindly advise.

Re: Splunk Forwarder Tail Fails

Kawtar — Mon, 19 Aug 2019 09:46:12 GMT

I have the same problem, but the version is not the problem, how can we resolve that plz ?