Getting Data In

Is it possible to reconfigure an existing universal forwarder to low-privilege mode?

lycollicott
Motivator

Is it possible reconfigure an existing universal forwarder to low privileged mode? We installed our UFs as local system and are being asked to change them to a user in low-privilege mode.

FritzWittwer
Contributor

We are running the Splunkforwarder as user splunk and group splunk on our Solaris and AIX systems. We then give Splunk access to the log files either by putting ACLs on this files, or by changing the group to Splunk and give group read access. It gets more tricky for apps with scripted inputs, usually we run a cron job outside of Splunk, e.g as the oracle user, which runs the scripts and writes its output to a file which is in turn read by Splunk.

0 Karma

Laurin_Green
New Member

I too am interested in knowing a technique for having Splunk Forwarder installed and running as non-root but still able to monitor root read only /var/adm/s* log files. Do you have a technique or a special Splunk privileged module or a startup parameter to enable reading root files?
In non-root mode, nothing in /opt/splunkforwarder is root owned, but do you have a special module that could be root owned to enable the reading of root only logs? I have to deploy SF to a couple of dozen servers and don't want to have them all running as root for a single log file on each server.

0 Karma

Laurin_Green
New Member

The following technique will work: 1) add a new special group; 2) adding SF to that group and only having SF in that group; 3) changing the group of the file with 600 permission to the new special group; 4) change the group access permission to add R to the file so that it is 640. This means that the file will be group readable by SF but since SF is the only userid in that group, only SF can read the file.
This will work so long as the permissions and group ownership do not change. In my case, the initial test worked, but the file got reverted, so this is not a workable solution.
I am still left with SF as non-root but unable to read root 600 files.
I think the Splunk Forwarder product should come with a special read module that can be rooted so that it can read root 600 files even though the rest of SF is installed non-root.

0 Karma

lycollicott
Motivator

I agree with jplumsdaine22 that adding the user to the adm group would probably work.

0 Karma

jplumsdaine22
Influencer

You can have splunk run as any user you please (see here http://docs.splunk.com/Documentation/Splunk/6.3.3/Installation/RunSplunkasadifferentornon-rootuser) but of course it will only be able to monitor files to which it has read access.

For example /var/log/syslog is usually only readable by root. if you have a non root user that can read the files you want go for it!

Alternatively I would suggest leaving splunk running under root but restrict incoming traffic to the management port to your deployment server, or if you don't use a deployment server restrict the management port to localhost only.

0 Karma

vinceskahan
Path Finder

Sounds pretty counter-intuitive to me, but thanks for the response.

Could somebody from Splunk Engineering chime in with their recommendation please ? I keep finding these edge cases where simple stuff is difficult/impossible/too-complicated in this product. Other than "don't do that" it would be nice to have a "do this instead" from Splunk Engineering 'officially' as a best practice. You know we will want to monitor our splunk server logs just like any other forwarding computer. What is the way we are supposed to do this while meeting your recommendation to not run the server processes as root ?

0 Karma

jplumsdaine22
Influencer

Have you tried just adding the splunk user to the group that has permissions to read /var/log/syslog ? eg adm on ubuntu

vinceskahan
Path Finder

Well, this is centos, so it's not adm, and many interesting files are mode 600 so group permissions wouldn't help. I'm reluctant to hack up the system to force things. Good idea though...

0 Karma

jplumsdaine22
Influencer

ls -l /var/log/syslog | cut -d' ' -f4

Is that root or something else?

0 Karma

vinceskahan
Path Finder

yes, on centos it's root:root - I know 'how' to hack the system up, I just don't want to do it.

Still awaiting an official Splunk chime-in on what they want us to do. Their silence on these kinds of things is a bit disconcerting.

0 Karma

vinceskahan
Path Finder

On a similar note - I know that Splunk recommends not running a 'server' (not forwarder) as root, but then it can't monitor itself for the same reason. I set up a new server to run as user splunk group splunk and couldn't monitor /var/log/messages due to permission denied.

Is there any reasonable way to let the non-privileged splunk processes monitor non-splunk-owned files ? Do I need to do something crazy like forwarding syslog to the indexer rather than using splunk ? Seems dumb when you get to that level of complexity when I can packet filter things and block input to the whole server.

Ideas ? Best practices that actually work ?

0 Karma

jplumsdaine22
Influencer

Well the spunk server needs to be accessible remotely hence the need to run it as a non root user. Just have a separate Universal Forwarder instance on your spunk server that runs as root and does not listen to external connections.

0 Karma

lycollicott
Motivator

I should clarify that I would prefer to not have to uninstall and then reinstall the UF.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!