topic Re: Root Can't Create /var/log files in Splunk Search

Root Can't Create /var/log files

heats — Mon, 28 Aug 2017 14:10:43 GMT

This is the first time this has come up:

When running the following command as root:

(10:07:49) root@servername:/opt/splunkforwarder/bin
--> ./splunk enable boot-start -user splunk

Warning: cannot create "/opt/splunkforwarder/var/log/splunk"

Warning: cannot create "/opt/splunkforwarder/var/log/introspection"
First-time-run has not finished. Ignore this error when previewing migration - exiting.

Any idea what could be causing this? Root permissions should have what's needed to create the var/log/ files

Re: Root Can't Create /var/log files

esix_splunk — Mon, 28 Aug 2017 14:28:20 GMT

Looks to me like there was a problem with the initial installation, or someone installed the forwarder as a different runtime user, and then restarted it as root.

You can try fixing permissions first, chown -R splunk:splunk /opt/splunkforwarder, then sudo to the splunk user and try running /opt/splunkforwarder/bin/splunk start.

See if you get those errors still. Count to that you can chown that to root:root, and run splunk start as root and see if you get the same errors.

Re: Root Can't Create /var/log files

jkat54 — Mon, 28 Aug 2017 14:44:55 GMT

Apparently someone has set the permissions on /opt/splunkforwarder so that only the owner of the directory can write files/folders under it.

For example:

chmod 700 /opt/splunkforwarder

would make it so that only the owner can read/write/execute it.

You can do the following to get around this but ultimately your permissions need to be fixed.

1st stop Splunk if it's running

/opt/splunkforwarder/bin/splunk stop

2nd, make root the owner of the Splunk dir:

chown -Rf root. /opt/splunkforwarder

3rd, run the same boot start command

/opt/splunkforwarder/bin/splunk enable boot-start -user splunk

4th, change ownership back to splunk user

chown -Rf splunk. /opt/splunkforwarder

5th, switch to Splunk user

su splunk

6th, restart Splunk

/opt/splunkforwarder/bin/splunk start

Re: Root Can't Create /var/log files

joshualemoine — Mon, 17 Jun 2019 16:57:06 GMT

Absolutely brilliant. I have been searching for this answer for quite some time. Thanks so much. The only extra step I had to do what kill the pid running splunk at the very end before su to splunk and starting splunk, b/c I couldn't stop splunk at the beginning of this sequence of commands due to the "unable to create introspection, var/log/splunk, and this was even as the root user! This all started from an improper clone of a server. Thanks again!

Re: Root Can't Create /var/log files

jkat54 — Tue, 18 Jun 2019 00:22:50 GMT

anytime, thanks for the upvote(s)!