Splunk Search

Root Can't Create /var/log files

heats
Explorer

This is the first time this has come up:

When running the following command as root:

(10:07:49) root@servername:/opt/splunkforwarder/bin
--> ./splunk enable boot-start -user splunk

Warning: cannot create "/opt/splunkforwarder/var/log/splunk"

Warning: cannot create "/opt/splunkforwarder/var/log/introspection"
First-time-run has not finished. Ignore this error when previewing migration - exiting.

Any idea what could be causing this? Root permissions should have what's needed to create the var/log/ files

Tags (2)
0 Karma
1 Solution

jkat54
SplunkTrust
SplunkTrust

Apparently someone has set the permissions on /opt/splunkforwarder so that only the owner of the directory can write files/folders under it.

For example:

chmod 700 /opt/splunkforwarder

would make it so that only the owner can read/write/execute it.

You can do the following to get around this but ultimately your permissions need to be fixed.

1st stop Splunk if it's running

/opt/splunkforwarder/bin/splunk stop

2nd, make root the owner of the Splunk dir:

chown -Rf root. /opt/splunkforwarder

3rd, run the same boot start command

/opt/splunkforwarder/bin/splunk enable boot-start -user splunk

4th, change ownership back to splunk user

chown -Rf splunk. /opt/splunkforwarder

5th, switch to Splunk user

su splunk

6th, restart Splunk

/opt/splunkforwarder/bin/splunk start

View solution in original post

jkat54
SplunkTrust
SplunkTrust

Apparently someone has set the permissions on /opt/splunkforwarder so that only the owner of the directory can write files/folders under it.

For example:

chmod 700 /opt/splunkforwarder

would make it so that only the owner can read/write/execute it.

You can do the following to get around this but ultimately your permissions need to be fixed.

1st stop Splunk if it's running

/opt/splunkforwarder/bin/splunk stop

2nd, make root the owner of the Splunk dir:

chown -Rf root. /opt/splunkforwarder

3rd, run the same boot start command

/opt/splunkforwarder/bin/splunk enable boot-start -user splunk

4th, change ownership back to splunk user

chown -Rf splunk. /opt/splunkforwarder

5th, switch to Splunk user

su splunk

6th, restart Splunk

/opt/splunkforwarder/bin/splunk start

joshualemoine
Path Finder

Absolutely brilliant. I have been searching for this answer for quite some time. Thanks so much. The only extra step I had to do what kill the pid running splunk at the very end before su to splunk and starting splunk, b/c I couldn't stop splunk at the beginning of this sequence of commands due to the "unable to create introspection, var/log/splunk, and this was even as the root user! This all started from an improper clone of a server. Thanks again!

jkat54
SplunkTrust
SplunkTrust

anytime, thanks for the upvote(s)!

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

Looks to me like there was a problem with the initial installation, or someone installed the forwarder as a different runtime user, and then restarted it as root.

You can try fixing permissions first, chown -R splunk:splunk /opt/splunkforwarder, then sudo to the splunk user and try running /opt/splunkforwarder/bin/splunk start.

See if you get those errors still. Count to that you can chown that to root:root, and run splunk start as root and see if you get the same errors.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...