Splunk Search

Root Can't Create /var/log files

heats
Explorer

This is the first time this has come up:

When running the following command as root:

(10:07:49) root@servername:/opt/splunkforwarder/bin
--> ./splunk enable boot-start -user splunk

Warning: cannot create "/opt/splunkforwarder/var/log/splunk"

Warning: cannot create "/opt/splunkforwarder/var/log/introspection"
First-time-run has not finished. Ignore this error when previewing migration - exiting.

Any idea what could be causing this? Root permissions should have what's needed to create the var/log/ files

Tags (2)
0 Karma
1 Solution

jkat54
SplunkTrust
SplunkTrust

Apparently someone has set the permissions on /opt/splunkforwarder so that only the owner of the directory can write files/folders under it.

For example:

chmod 700 /opt/splunkforwarder

would make it so that only the owner can read/write/execute it.

You can do the following to get around this but ultimately your permissions need to be fixed.

1st stop Splunk if it's running

/opt/splunkforwarder/bin/splunk stop

2nd, make root the owner of the Splunk dir:

chown -Rf root. /opt/splunkforwarder

3rd, run the same boot start command

/opt/splunkforwarder/bin/splunk enable boot-start -user splunk

4th, change ownership back to splunk user

chown -Rf splunk. /opt/splunkforwarder

5th, switch to Splunk user

su splunk

6th, restart Splunk

/opt/splunkforwarder/bin/splunk start

View solution in original post

jkat54
SplunkTrust
SplunkTrust

Apparently someone has set the permissions on /opt/splunkforwarder so that only the owner of the directory can write files/folders under it.

For example:

chmod 700 /opt/splunkforwarder

would make it so that only the owner can read/write/execute it.

You can do the following to get around this but ultimately your permissions need to be fixed.

1st stop Splunk if it's running

/opt/splunkforwarder/bin/splunk stop

2nd, make root the owner of the Splunk dir:

chown -Rf root. /opt/splunkforwarder

3rd, run the same boot start command

/opt/splunkforwarder/bin/splunk enable boot-start -user splunk

4th, change ownership back to splunk user

chown -Rf splunk. /opt/splunkforwarder

5th, switch to Splunk user

su splunk

6th, restart Splunk

/opt/splunkforwarder/bin/splunk start

joshualemoine
Path Finder

Absolutely brilliant. I have been searching for this answer for quite some time. Thanks so much. The only extra step I had to do what kill the pid running splunk at the very end before su to splunk and starting splunk, b/c I couldn't stop splunk at the beginning of this sequence of commands due to the "unable to create introspection, var/log/splunk, and this was even as the root user! This all started from an improper clone of a server. Thanks again!

jkat54
SplunkTrust
SplunkTrust

anytime, thanks for the upvote(s)!

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

Looks to me like there was a problem with the initial installation, or someone installed the forwarder as a different runtime user, and then restarted it as root.

You can try fixing permissions first, chown -R splunk:splunk /opt/splunkforwarder, then sudo to the splunk user and try running /opt/splunkforwarder/bin/splunk start.

See if you get those errors still. Count to that you can chown that to root:root, and run splunk start as root and see if you get the same errors.

0 Karma
Get Updates on the Splunk Community!

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability As businesses scale ...