topic Re: Splunk Permission Error in Deployment Architecture

Why are we experiencing this Splunk Permission Error

jbender72 — Wed, 27 Apr 2022 15:39:57 GMT

Hello,

Noticed my Indexer was down and I could not sign in. Went to restart splunk as sudo then root user and got this error:

Splunk is unable to write to the directory /opt/splunk and therefore will not run. Please check for appropriate permissions on this directory and its contents as necessary.

Checked what I could, I cannot come up with a solution. Did not find much on the Internet. Please help.

Re: Splunk Permission Error

nickhills — Tue, 15 Dec 2020 16:09:42 GMT

Was Splunk configured to run as "root"?

If you configured Splunk to run as "splunk" (recommended) you should not start it as root. If you do, it may mess about with the folder permissions, which means the next time you start it as "splunk" you get permission errors.

If this is what happened, you should stop Splunk (if running) then (assuming linux)

sudo chown -R splunk:splunk /opt/splunk

Then start Splunk as "splunk" - probably easier to reboot the system and let the Splunk boot-start process handle it.

Re: Splunk Permission Error

jbender72 — Tue, 15 Dec 2020 16:42:36 GMT

I don't know if this is in the correct area. I am getting this: invalid user.

azureuser@cb-spl-in1-p ~]$ sudo chown -R splunk:splunk /opt/splunk
chown: invalid user: ‘splunk:splunk’

Am I placing the command in the incorrect place?

Re: Splunk Permission Error

nickhills — Tue, 15 Dec 2020 17:25:41 GMT

Thats odd.

Can you run "sudo id splunk"

It should return something like

uid=xxx(splunk) gid=xxx(splunk) group=xxx(splunk)

or does it report
"splunk" no such user

Re: Splunk Permission Error

jbender72 — Tue, 15 Dec 2020 17:39:04 GMT

It is giving no such user

Re: Splunk Permission Error

nickhills — Tue, 15 Dec 2020 18:26:48 GMT

How was splunk installed? rpm/deb or from the tar.gz?

Who owns files in /opt/splunk?

Re: Splunk Permission Error

jbender72 — Tue, 15 Dec 2020 20:53:48 GMT

as tar

Re: Splunk Permission Error

nickhills — Tue, 15 Dec 2020 20:59:46 GMT

In that case, unless you set up the splunk user by hand, it wont exist.

At a guess you were therefore running Splunk as root.
If root has permission issues then you probably have bigger problems!

Can you access the contents of /opt/splunk?

Re: Splunk Permission Error

jbender72 — Tue, 15 Dec 2020 21:00:07 GMT

I tried to set the azureuser as owner of the files who owns splunk. I chmod 777 and it appeared to work.

[azureuser@cb-spl-in1-p splunk]$ ls -al
total 3200
drwxrwxrwx. 10 azureuser azureuser 237 Dec 15 16:48 .
drwxrwxrwx. 3 root root 20 Nov 13 20:09 ..
drwxrwxrwx. 4 azureuser azureuser 4096 Nov 12 15:37 bin
-rwxrwxrwx. 1 azureuser azureuser 57 Nov 12 15:37 copyright.txt
drwxrwxrwx. 16 azureuser azureuser 4096 Dec 15 15:34 etc
drwxrwxrwx. 4 azureuser azureuser 62 Nov 12 15:37 include
drwxrwxrwx. 8 azureuser azureuser 4096 Nov 12 15:37 lib
-rwxrwxrwx. 1 azureuser azureuser 85709 Nov 12 15:37 license-eula.txt
drwxrwxrwx. 3 azureuser azureuser 58 Nov 12 15:37 openssl
-rwxrwxrwx. 1 azureuser azureuser 844 Nov 12 15:37 README-splunk.txt
drwxrwxrwx. 4 azureuser azureuser 108 Nov 12 15:37 share
-rwxrwxrwx. 1 azureuser azureuser 3168712 Nov 12 15:37 splunk-8.1.0-f57c09e87251-linux-2.6-x86_64-manifest
drwxrwxrwx. 2 azureuser azureuser 54 Nov 12 15:37 swidtag
drwxrwxrwx. 6 azureuser azureuser 52 Nov 12 15:37 var
[azureuser@cb-spl-in1-p splunk]$

Re: Splunk Permission Error

nickhills — Tue, 15 Dec 2020 21:24:46 GMT

I am pleased you have it working, but for the record, chmod'ding to 777 is not a sensible fix for a production system. 🙂

Re: Splunk Permission Error

jbender72 — Tue, 15 Dec 2020 22:53:16 GMT

Yes I can access the contents, just no splunk user anymore?

Re: Splunk Permission Error

Ocelot — Wed, 13 Apr 2022 17:09:33 GMT

I came across this error today. It turned out to be a disk space issue. Provisioning more storage allowed Splunk to start without issue.

Re: Splunk Permission Error

Jumy — Wed, 27 Apr 2022 10:25:56 GMT

I am having same issue, how do I provision for the disk space issue? Thanks

chown: changing ownership of '/opt/splunk/.bash_history': Read-only file system
chown: changing ownership of '/opt/splunk': Read-only file system