Security

How to resolve [Errno 13] Permission denied?

julian0125
Explorer

Hello friend,

I've got the next issue trying to run ./splunk start or status. How can i fix it? i think it is a user permission issue.

[root@cerr500810 bin]# ./splunk start

Warning: cannot create "/monitoreo/splunk/var/log/splunk"

Warning: cannot create "/monitoreo/splunk/var/log/introspection"
Pid file "/monitoreo/splunk/var/run/splunk/splunkd.pid" unreadable.: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/Splunk_TA_nix/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/TA-tripwire_enterprise/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/launcher/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/lookup_editor/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/search/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/splunk_instrumentation/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/system/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/learned/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/Splunk_TA_nix/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/TA-tripwire_enterprise/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/launcher/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/lookup_editor/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/search/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/splunk_instrumentation/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/system/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/learned/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/Splunk_TA_nix/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/TA-tripwire_enterprise/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/launcher/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/lookup_editor/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/search/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/splunk_instrumentation/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/system/metadata/local.meta: Permission denied
Cannot initialize: /monitoreo/splunk/etc/apps/learned/metadata/local.meta: Permission denied
Pid file "/monitoreo/splunk/var/run/splunk/splunkd.pid" unreadable.: Permission denied
Pid file "/monitoreo/splunk/var/run/splunk/splunkd.pid" unreadable.: Permission denied
Labels (1)
Tags (1)
0 Karma

arjunpkishore5
Motivator

I see that you already are root. This is bad practice. Never use root to start the service.

Next, check the permissions on the file system as suggested by @woodcock

If the permissions are set up correctly, check if the disk is mounted properly. I have seen instances where the disk is set to read-only mode accidentally by the linux admin. This makes startup or usage impossible since splunk will be unable to create files.

aamer86
Path Finder

I see Splunk is not installed under the default directory
try running this

you have to make sure Splunk is running from
/opt/splunk/bin/splunk start

1- change the splunk home directory to /opt/splunk
2- Run (using the root user) /opt/splunk/bin/splunk stop
3- chown -R splunk /opt/
4- sudo su splunk
5- /opt/splunk/bin/splunk start

0 Karma

arjunpkishore5
Motivator

Hi @aamer86

This is incorrect. Default directory is just that - a default directory and not a mandatory directory. It's possible to change the base directory and have splunk running by updating the value of $SPLUNK_HOME

0 Karma

woodcock
Esteemed Legend

This happens when you have started splunk as user root and then later try to start it as the correct non- root user (usually splunk). To fix, do this:

AS USER root:

/opt/splunk/bin/splunk start
chown -R splunk: $SPLUNK_HOME
service splunk start

nnimbe1
Path Finder

@woodcock this was the same issue which i was facing...and now using your steps it resolved thanks a lot

0 Karma

andhika_pratama
Explorer

you might be root, but the Splunk owner is another user.
1. command ls -l or ll to know who the owner is
2. switch to Splunk user

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Try using this command instead

sudo service splunk restart

It seems the splunk is running with different user and you're trying to restart it with user root. The above service command will ensure Splunk service will restart with the account it's setup to run with.

0 Karma

julian0125
Explorer

hello @somesoni2

i tried with the command "sudo service splunk restart" but it shows the next:

[root@cerr500810 bin]# sudo service splunk restart
Redirecting to /bin/systemctl restart splunk.service
Job for splunk.service failed because the control process exited with error code. See "systemctl status splunk.service" and "journalctl -xe" for details.
[root@cerr500810 bin]# sudo service splunk restart
Redirecting to /bin/systemctl restart splunk.service
Job for splunk.service failed because the control process exited with error code. See "systemctl status splunk.service" and "journalctl -xe" for details.
[root@cerr500810 bin]# sudo service splunk start
Redirecting to /bin/systemctl start splunk.service
Job for splunk.service failed because the control process exited with error code. See "systemctl status splunk.service" and "journalctl -xe" for details.
[root@cerr500810 bin]# sudo service splunk status
Redirecting to /bin/systemctl status splunk.service
● splunk.service - splunk Service , para monitoreo de Seguridad
Loaded: loaded (/etc/systemd/system/splunk.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Thu 2019-02-21 20:28:16 UTC; 4s ago
Process: 24782 ExecStart=/usr/local/sbin/splunk.sh (code=exited, status=2)
Main PID: 24782 (code=exited, status=2)

Feb 21 20:28:15 cerr500810 systemd[1]: Starting splunk Service , para monitoreo de Seguridad...
Feb 21 20:28:16 cerr500810 systemd[1]: splunk.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Feb 21 20:28:16 cerr500810 systemd[1]: Failed to start splunk Service , para monitoreo de Seguridad.
Feb 21 20:28:16 cerr500810 systemd[1]: Unit splunk.service entered failed state.
Feb 21 20:28:16 cerr500810 systemd[1]: splunk.service failed.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Can you run this and see under what user Splunkd service is running?

ps -ef  | grep splunkd | grep start | grep -v grep
0 Karma

julian0125
Explorer

Sure! i've got this

[segemer@cerr500810 system]$ ps -ef | grep splunkd | grep start | grep -v grep
root 7805 1 0 Feb13 ? 00:29:16 splunkd -p 8089 start
root 7824 7805 0 Feb13 ? 00:00:00 [splunkd pid=7805] splunkd -p 8089 start [process-runner]
[segemer@cerr500810 system]$

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Looks like at some point of time, Splunk started with wrong user. It's currently running as root. Do you always run Splunk as root OR use a non-root splunk user account?

Also, who owns Splunk file system (run ls -ltr /monitoreo/splunk)?
What's content of attribute SPLUNK_OS_USER in file /monitoreo/splunk/etc/splunk-launch.conf ?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...