Security

How to get "splunk" user to read "root" user-owned files?

redc
Builder

Our general policy is to not run applications on our servers as the "root" user. However, some log files get written with the read/write access to only the owner (not to any group, and not to others), and the owner is always "root".

When attempting to add a monitor stanza on this file in the inputs.conf for the forwarder, I get the following error:

Insufficient permissions to read file='/file/location/filename' (hint: Permission denied).

(The file in question is an SMTP maillog.)

Permissions on the file:

-rw------- root root

As I understand it, one of the perks/features of Splunk is that the non-root user (in this case, "splunk") should be able to be assigned read permissions to all files, even ones that are owned by root.

Changing the permissions on the file won't work because each night when the log rolls over, the permissions will get reset.

What's the way to get around this? If I remember correctly from things I've heard (and a few of the other Answers posts allude to this), there's some file on the host that can be modified to add the "splunk" user and grant read-only access to these root-owned files, but I don't know/understand the specifics of how to do that.

The forwarder is running on a Linux CentOS 5.11 system. The "splunk" user is a local user.

1 Solution

dwaddle
SplunkTrust
SplunkTrust

Giving sudo rights for the user 'splunk' to start splunk as 'root' doesn't help here. So let's skip that idea entirely.

Options are:

  1. Traditional Unix permissions model
  2. POSIX ACLs
  3. Something magical

In the traditional Unix permissions model, the kernel uses the user/group/other bits to decide if the process in question has the rights to read the file. You could set your /var/log directory (and subdirs) to be owned by group 'syslog' and make the 'syslog' group a supplemental group for the user 'splunk'. From there, you'll need to chmod g+s /var/log (and any other subdirs of /var/log) in order to force the files to be owned by the supplemental group. (A setgid on a directory makes the kernel create any new files in that directory with the group-ownership set to the group-ownership of the directory itself). You'll also need to change the umask for files written by the syslog server to something more like 007 instead of 022 or 077. That way, the files will have group-read. Between the umask enabling group-read, the setgid, and the supplemental group, Splunk should be able to read the files.

If you use POSIX ACLs, you can define additional privs above/beyond the standard Unix permissions model. So you could give the user 'splunk' specific read access to these files. As mentioned elsewhere in other answers here, the setfacl command is the route here. The problem here is that few tools properly understand POSIX ACLs and you can run into trouble where newly created files (post-logrotate) may not get the right ACL applied every time. Another problem is that even though these have POSIX in their name, the userspace command set and semantics for configuring them is wonky from OS to OS. Linux, AIX, and Solaris all do it slightly differently. This works, but expect to put some energy into making it work.

As far as magical things, there's the concept of "capabilities". Both Linux and Solaris have the concept of "I can give this process environment something greater than normal user, but less than full root." POSIX introduced the idea of "CAP_DAC_READ_SEARCH" which is basically "read only root" -- the kernel will not do checks to see if your process has access to read any file. With this capability flag, Splunk can transparently read any and all files as if it were root, but the ability to write files is kept per normal. With Solaris, this is available when launching Splunk via SMF. With Linux, there is setcap. But, as of Splunk 6.2.3, the setcap approach on Linux DOES NOT WORK. I won't go into details, because they are ugly. I'm hoping to get a support case out there soon and see if it can be made to work...

View solution in original post

woodcock
Esteemed Legend

The last solution does not require you to modify the application; unless the application specifies/enforces the permissions (only one way to find out), this will work because it is modifying the OS, not the application. You have nothing to lose by trying.

0 Karma

a212830
Champion

If it's syslog, what daemon are you running? You should be able to modify the permissions of the output file via the syslog daemon, not your app.

0 Karma

redc
Builder

This log is being written by custom code in our application, not syslog.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...