Hi,
my splunk is running as splunk user on a linux system where the admin has secured the OS by using hidepid=1 on /proc (see https://ubuntuforums.org/showthread.php?t=2173093 and https://www.kernel.org/doc/Documentation/filesystems/proc.txt)
As a consequence, splunkd.log is filled with these error messages :
ERROR IntrospectionGenerator:resource_usage - RU - Fail to readlink(2) /proc/nnnn/exe: Operation not permitted where nnnn is a pid from a process not run by splunk
This is repeated for each pid so generate a lot of noise.
I would like to tell Introspection to only look at it's own pid in that case or not produce error message for this.
Any idea how to do this ?
Hi,
you can also add the splunk group gid to the fstab ($ id splunk_user) :
proc /proc proc rw,nosuid,nodev,noexec,relatime,gid=<splunk_gid>,hidepid=1 0 0
According to man proc :
gid=gid (since Linux 3.3) Specifies the ID of a group whose members are authorized to learn process information otherwise prohibited by hidepid (i.e., users in this group behave as though /proc was mounted with hidepid=0). This group should be used instead of approaches such as putting nonroot users into the sudoers(5) file.
As a workaround, I completely disabled the generator for resource usage
in server.conf
[introspection:generator:resource_usage]
disabled=true
this stop the error message flood but that will also disable all related stats in the monitoring console....