All Apps and Add-ons

Linux Auditd: What is the best way to make /var/log/audit/audit.log accessible to a non-root Splunk forwarder?

dominiquevocat
Motivator

Hey,

Ilike the Linux Auditd app and the basic idea. Now the audit.log is owned by root (kinda logic I guess) and the Splunk forwarder does not run as root (kinda logic i guess). So what would be the best way to handle this?

1 Solution

dominiquevocat
Motivator

setacl is your friend:

setfacl -m g:splunk:r /var/log/audit/audit.log

for instance... 🙂

View solution in original post

fharding
Explorer

My two cents on this. I tried using setfacl in our environment, but auditd kept changing the group permissions on audit.log when it rolls the log, which in turn jacks up the setfacl permissions. I tried running incrond as a watchdog to automatically change the permissions back if auditd messed them up, but found that incrond would crash after a while.

Here is the solution I settled on. This is in the context of RHEL7/CentOS7, running Splunk as non-root user "splunk". From some of my old notes:

#the following steps make the audit.log readable by splunk
#add adm group if dne, add splunk user to adm group, set log_group to adm in auditd.conf, fix dir perms, restart auditd
id -g adm &>/dev/null || groupadd adm
usermod -a -G adm splunk
sed -i.bak 's/log_group = .*/log_group = adm/g' /etc/audit/auditd.conf
chgrp -R adm /var/log/audit/
chmod 0750 /var/log/audit/
chmod 0640 /var/log/audit/*
service auditd restart

dominiquevocat
Motivator

setacl is your friend:

setfacl -m g:splunk:r /var/log/audit/audit.log

for instance... 🙂

nsommars
Explorer

For /var/log/audit/audit.log it is better to use pbunts solution (answer below).
Posix file ACLs tend to get tricky when there is no automatic way to apply the acl when the daemon rotates the log file.
(yes, you could have a cron script that applies the ACL at short intervals...but that is quite an ugly solution)

For other log files, rotated by logrotate, file ACLs is the way to go when one can re-apply the ACL in postrotate.

Intermediate
Path Finder
  1. Do NOT do this on an Ubuntu (possibly Debian?) based system. auditd will fail to restart if the permissions of the audit log have been tampered with in any way.
  2. On all Linux distros OS audit tools (like "ausearch") will complain if ACLs are added to the audit log.
  3. As @nsommars points out it does not deal with log rotation.

I would suggest that @pbunts has the pragmatic solution, with @doksu's SELinux policy being the perfect but highly difficult option.

0 Karma

pbunts
Engager

modify /etc/audit/auditd.conf and change log_group from root to splunk. restart auditd and now /var/log/audit.log will have group read and be set to the splunk group.

doksu
SplunkTrust
SplunkTrust

All the complexities and workarounds associated with not running Splunk as root can be avoided by simply running it as root but securely using SELinux: https://github.com/doksu/selinux_policy_for_splunk

There are also a number of other added benefits with this approach. When the SELinux policy is applied to Splunk, it protects it from other services that may have been compromised and vice versa. Furthermore, in a PaaS environment, you can use this SELinux policy to easily apply RBACs, delegating only the permissions required to manage the Splunk service as root. I haven't yet formally published this guide, but here's the working draft: https://github.com/doksu/selinux_policy_for_splunk/issues/5

Intermediate
Path Finder

@doksu it might be worth you putting your thoughts in this RH solution too: https://access.redhat.com/solutions/905243 as it's the same issue.

0 Karma

doksu
SplunkTrust
SplunkTrust

Thanks very much @Intermediate , will do. 🙂

0 Karma

Intermediate
Path Finder

I agree this is a great idea for RHEL or CentOS - but the original poster doesn't say which Linux distro they are using.
I'm not convinced this solution would be ideal using a distribution which uses Apparmor (even though Apparmor profiles are simpler to write) because it doesn't provide the same level of confinement.

0 Karma

doksu
SplunkTrust
SplunkTrust

Also, be sure to collect the audit log events using a normal inputs.conf monitor stanza rather than the nix app's scripted input. The nix app's scripted input modifies audit events in resolving uids; the script doesn't always work well and its functionality is superseded by the Linux Auditd app's uid resolution at search time (best practice).

0 Karma

dominiquevocat
Motivator

This sounds like the best way to move forward. I hope you wil publish your guide in a finalized 1.0 version and will propose this in our organisation.

0 Karma

doksu
SplunkTrust
SplunkTrust

I've appended an RBACs implementation guide to the README: https://github.com/doksu/selinux_policy_for_splunk

0 Karma

frmaasdam
Path Finder

Or to add an another solution. You can add an acl on that particular file.

0 Karma

muebel
SplunkTrust
SplunkTrust

Hello sir! Couple ideas, you could add the splunk user to a group, and set group permissions on the log so that the user could read it. Also, you could forward out the events via syslog to a system that does have a forwarder running in such a way as it can read the logs.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...