Getting Data In

Splunk Forwarder Tail Fails

OldManEd
Builder

I have several forwarders that are release 4.3.2. The issue is that the log files they are configured to send to my indexers periodically rotate. For some reason Splunk will not send any more data when this happens. But, if I restart the forwarder, data starts coming in with no problems. The strange thing is that this only happens on some of the forwarders.

File permissions were checked and found to be OK.

The related errors in the splunkd.log file are;

06-14-2014 08:00:08.608 -0600 ERROR TailingProcessor - Ignoring path due to: failed to open for checksum: '/var/tmp/xxx/xxx.log' (No such file or directory)

06-14-2014 08:00:09.911 -0600 ERROR TailingProcessor - Unable to resolve path for symlink: /var/tmp/xxx/xxx.log.

Any ideas?

1 Solution

the_wolverine
Champion

Splunk bugs go unreported (or undetected) often. I suggest upgrading to a later version of the forwarder, at least version 4.3.4 or 5.0.7 to see if you still encounter the issue.

View solution in original post

scc00
Contributor

So I had the same issue. The fix was upping the permissions to the symlink. So check to see if the user for your splunk install can get to the full location and increase the permissions accordingly.

0 Karma

kamaljagga
Path Finder

I am facing the same issue and my forwarder version is 4.2.6 and forwarder runs as root. Thinking of doing a clean re-install of forwarder.
If anyone found any other solution. Kindly advise.

the_wolverine
Champion

Splunk bugs go unreported (or undetected) often. I suggest upgrading to a later version of the forwarder, at least version 4.3.4 or 5.0.7 to see if you still encounter the issue.

Kawtar
Path Finder

I have the same problem, but the version is not the problem, how can we resolve that plz ?

0 Karma

abonuccelli_spl
Splunk Employee
Splunk Employee

06-14-2014 08:00:09.911 -0600 ERROR
TailingProcessor - Unable to resolve
path for symlink:
/var/tmp/xxx/xxx.log.

can you actually resolve that symlink using ls?

tail /path/to/symlink

0 Karma

OldManEd
Builder

Yes. The link is "/var/tmp/xxx/xxx.log -> /usr/xxx/yyy/AAA/work/xxx_aaa_99.log". From what I understand, the target of the link, in this case "/usr/xxx/yyy/AAA/work/xxx_aaa_99.log", is somehow "processed" and there is a period of about 1.0 to 1.5 seconds where it doesn't exist. But once the process is complete, everything functions correctly except the forwarder doesn't continue to tail the new file.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...