Hi,
Splunk UF is setup to read files from particular directory. It reads files normally for few minutes, but suddenly it throws an error
WARN FilesystemChangeWatcher - error reading directory "xxxx/xxxx/xxx": Permission denied
Again when we reinstall splunk it reads normally and stops after few minutes showing permission denied.
May I know, Is there any reason that it suddenly shows permission denied error after reading half of the file.
Is there any possibility that when a file is written by some other process , It blocks other process to read the file?
have you checked the permission for that directory and if the user running splunk had read permissions for that directory?
yes there are sufficient permissions. We could sudo to splunk user and open files to read
This a UF on windows or linux? What's OS / splunk UF version?
UF on linux. Redhat 7, splunk UF 6.5.7
Are the ones that are failing in /var/log ? Can you confirm if you've set permissions there manually or have configured a custom umask on them that will allow "other" or a group the splunk user is part of to read/execute on directories and read files? Also can you confirm that you're not manually setting permissions there when you reinstall splunk, it "just works" after the reinstall.
Splunk user is the part of the same group the file is part of
Ex:
drwxrwx--- user abc wer.log
Splunk user is part of abc group who has read permissions.
I haven't set any Manuel permissions
Can you change to the splunk user (su - splunk) when the error occurs and less <file.log>
without issue?
Yes, I can sudo to splunk user and view the file.