Splunk Dev

How do I prevent Introspection Generator to read information about non splunk process when hidepid activated on /proc ?

maraman_splunk
Splunk Employee
Splunk Employee

Hi,

my splunk is running as splunk user on a linux system where the admin has secured the OS by using hidepid=1 on /proc (see https://ubuntuforums.org/showthread.php?t=2173093 and https://www.kernel.org/doc/Documentation/filesystems/proc.txt)

As a consequence, splunkd.log is filled with these error messages :
ERROR IntrospectionGenerator:resource_usage - RU - Fail to readlink(2) /proc/nnnn/exe: Operation not permitted where nnnn is a pid from a process not run by splunk
This is repeated for each pid so generate a lot of noise.

I would like to tell Introspection to only look at it's own pid in that case or not produce error message for this.

Any idea how to do this ?

0 Karma

ipfyx
Engager

Hi,

you can also add the splunk group gid to the fstab ($ id splunk_user) :
proc /proc proc rw,nosuid,nodev,noexec,relatime,gid=<splunk_gid>,hidepid=1 0 0

According to man proc :

       gid=gid (since Linux 3.3)
              Specifies the ID of a group whose members are authorized
              to learn process information otherwise prohibited by
              hidepid (i.e., users in this group behave as though /proc
              was mounted with hidepid=0).  This group should be used
              instead of approaches such as putting nonroot users into
              the sudoers(5) file.

 

maraman_splunk
Splunk Employee
Splunk Employee

As a workaround, I completely disabled the generator for resource usage
in server.conf

[introspection:generator:resource_usage]
disabled=true

this stop the error message flood but that will also disable all related stats in the monitoring console....

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...