topic Re: Permission denied in Getting Data In

Permission denied

tattersp — Wed, 25 Jan 2017 14:37:52 GMT

I am running Splunk enterprise 6.3.1 and universal forwarder. We deploy the universal forwarder onto a Linux machine it runs under the account of Splunk.
Splunk is started with the account Splunk and that has the following
uid=880(splunk) gid=880(splunk) groups=880(splunk),600(dba),1201(buildgrp)

But it appears that it can not see directories or files owned by the dba group ie
drwxr-x--- 8 oracle dba 4096 Jan 24 22:15 par-01
drwxr-xr-x 3 oracle dba 4096 Jan 24 21:15 par-02
drwxr-xr-x 3 oracle dba 4096 Jan 24 21:15 par-03
drwxr-xr-x 3 oracle dba 4096 Jan 24 21:15 par-04
it can see par-02 to par-04 but not par-01

Re: Permission denied

DalJeanis — Wed, 25 Jan 2017 14:49:51 GMT

In any case, it's not a splunk issue.

You will need to check with your network security folks to see why the splunk ID is refused access to that directory. Chances are pretty good that this is a security feature, where your splunk account does not have the appropriate permissions.

This MAY be organizational security division-of-powers rules, which often say, "you can either have access A or access B, but not both." If that is the case, then you need to arrange to have the "hidden" directories mirrored somewhere you can access, or get a waiver, or do without.

FYI - the data you posted indicates that the splunk account does not belong to any group that has permissions to the 01 directory. The permission strings are in 3 sets of 3 indicators - "rwx" for each of user, group and everybody-else. the first item of each triple is the ability to read, the second is the ability to write, and the third is the ability to execute. A dash in a position means that ability is denied to that person or group.

Re: Permission denied

tattersp — Wed, 25 Jan 2017 14:59:55 GMT

So here is the issue. We set up an account in Linux that can access these files when you are logged on to the box as Splunk. Permissions are correct. But when the Splunk Universal forwarder tries to access them it gets permission denied.

01-25-2017 14:17:55.326 +0000 WARN FilesystemChangeWatcher - error reading directory "/user_projects/domains/pgcprd/servers/pgc-01": Permission denied

I have found a work currently the Splunk account has
groups=880(splunk),600(dba),1201(buildgrp)
but if i change it to
group=600(dba),1201(buildgrp),880(splunk)
it works fine.

Is there an issue with Splunk being a member of more than one group?

Re: Permission denied

DalJeanis — Wed, 25 Jan 2017 15:06:17 GMT

Hmmm. That seems like the local access protocol is only looking at the first group... Under a Windows machine, that could be the inheritance on the folder. Not sure if that's the same on Linux boxes.

With your security set this second way, try to access a directory that is owned by splunk and restricted to splunk itself. See if that gets blocked. Then try the same with buildgrp. If you can access splunk and buildgrp, then it's something about the dba folder or group, and if you cannot, then it's something intrinsic to the way that security is implemented on your boxes. May be a feature, may be a bug.

Re: Permission denied

hgrow — Wed, 25 Jan 2017 15:27:36 GMT

Hi there,

I've seen similar behaviour with old splunk universal forwarders with a bug in setting correct guids for the splunk process on startup.

Let your forwarder run ID with a scriped input:

../bin/id.sh

id

../inputs.conf

[script://./bin/id.sh]
index = something_tmp
interval = 3600
sourcetype = id

Compare the out with running > id as local splunk user on the OS.

Maybe it gives your more insights on whats going on