Installation

Linux UF boot-start issues and non-impacting errors on 9.0.0

_joe
Communicator

Is anyone else running into boot-start/permissions issues with the 9.0.0 UF running on Linux using init.d scripts for bootstart?

Warning: Attempting to revert the SPLUNK_HOME ownership
Warning: Executing "chown -R splunk /opt/splunkforwarder"


I am also finding that "./splunk disable boot-start" does not correctly remove the /etc/init.d/splunk script and, contrary to documentation, splunk UF 9.0.0 uses systemd as default.
https://docs.splunk.com/Documentation/Splunk/9.0.0/Admin/ConfigureSplunktostartatboottime

Also systemd scripts seem to fail getting the permissions needed even when trying to enable-boot as root. A key error I am seeing is "Failed to create the unit file" when running the install. But it seems to be a total fail.

 

 

## When upgrading (from 8.2.5)
runuser -l splunk -c "/opt/splunkforwarder/bin/splunk stop"
tar -xzvf /tmp/splunkforwarder-9.0.0-6818ac46f2ec-Linux-x86_64.tgz -C /opt
chown -R splunk:splunk /opt/splunkforwarder/
runuser -l splunk -c "/opt/splunkforwarder/bin/splunk start --accept-license --answer-yes --no-prompt"
runuser -l splunk -c "/opt/splunkforwarder/bin/splunk status"
Warning: Attempting to revert the SPLUNK_HOME ownership
Warning: Executing "chown -R splunk /opt/splunkforwarder"
(NOTE: Seems to be non-impacting)

 

### When doing a new install
tar -xzvf /tmp/splunkforwarder-9.0.0-6818ac46f2ec-Linux-x86_64.tgz -C /opt
chown -R splunk:splunk /opt/splunkforwarder
[root]#  sudo -H -u splunk /opt/splunkforwarder/bin/splunk start --accept-license --answer-yes --no-prompt
Warning: Attempting to revert the SPLUNK_HOME ownership
Warning: Executing "chown -R splunk /opt/splunkforwarder"

This appears to be your first time running this version of Splunk.

IMPORTANT: Because an admin password was not provided, the admin user
 will not be created. You will have to set up an admin username/password
 later using user-seed.conf.
Creating unit file...
Current splunk is running as non root, which cannot operate systemd unit files.
Please create it manually by 'sudo splunk enable boot-start' later.
Failed to create the unit file. Please do it manually later.


Splunk> Now with more code!

sudo -H -u splunk /opt/splunkforwarder/bin/splunk status
Warning: Attempting to revert the SPLUNK_HOME ownership
Warning: Executing "chown -R splunk /opt/splunkforwarder"
splunkd is running (PID: 3132350).
splunk helpers are running (PIDs: 3132354).

 

# sudo -H -u splunk /opt/splunkforwarder/bin/splunk stop
Warning: Attempting to revert the SPLUNK_HOME ownership
Warning: Executing "chown -R splunk /opt/splunkforwarder"
Stopping splunkd...
Shutting down.  Please wait, as this may take a few minutes.
.                                                          [  OK  ]
Stopping splunk helpers...
                                                           [  OK  ]
Done.
# /opt/splunkforwarder/bin/splunk enable boot-start -user splunk
Systemd unit file installed by user at /etc/systemd/system/SplunkForwarder.service.
Configured as systemd managed service.


systemctl start SplunkForwarder.service
Job for SplunkForwarder.service failed because the control process exited with error code.
See "systemctl status SplunkForwarder.service" and "journalctl -xe" for details.

systemctl status SplunkForwarder.service
● SplunkForwarder.service - Systemd service file for Splunk, generated by 'splunk enable boot-start'
   Loaded: loaded (/etc/systemd/system/SplunkForwarder.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2022-06-21 12:58:55 UTC; 27s ago
  Process: 3141480 ExecStartPost=/bin/bash -c chown -R splunk:splunk /sys/fs/cgroup/memory/system.slice/SplunkForwarder.service (code=exited, status=0/SUCCES>
  Process: 3141478 ExecStartPost=/bin/bash -c chown -R splunk:splunk /sys/fs/cgroup/cpu/system.slice/SplunkForwarder.service (code=exited, status=0/SUCCESS)
  Process: 3141477 ExecStart=/opt/splunkforwarder/bin/splunk _internal_launch_under_systemd (code=exited, status=203/EXEC)
  Process: 3141475 ExecStartPre=/bin/bash -c chown -R splunk:splunk /opt/splunkforwarder (code=exited, status=0/SUCCESS)
 Main PID: 3141477 (code=exited, status=203/EXEC)

Jun 21 12:58:55 <host> systemd[1]: SplunkForwarder.service: Failed with result 'exit-code'.
Jun 21 12:58:55 <host> systemd[1]: Failed to start Systemd service file for Splunk, generated by 'splunk enable boot-start'.
Jun 21 12:58:55 <host> systemd[1]: SplunkForwarder.service: Service RestartSec=100ms expired, scheduling restart.
Jun 21 12:58:55 <host> systemd[1]: SplunkForwarder.service: Scheduled restart job, restart counter is at 5.
Jun 21 12:58:55 <host> systemd[1]: Stopped Systemd service file for Splunk, generated by 'splunk enable boot-start'.
Jun 21 12:58:55 <host> systemd[1]: SplunkForwarder.service: Start request repeated too quickly.
Jun 21 12:58:55 <host> systemd[1]: SplunkForwarder.service: Failed with result 'exit-code'.
Jun 21 12:58:55 <host> systemd[1]: Failed to start Systemd service file for Splunk, generated by 'splunk enable boot-start'.



Labels (2)
1 Solution

_joe
Communicator

This warning got added to "known issues" for 9.0.0

Date filed Issue number Description

2022-06-23SPL-226019Warning appears in the universal forwarder whenever any spl command is run: Warning: Attempting to revert the SPLUNK_HOME ownership Warning: Executing "chown -R splunk /opt/splunkforwarder". This warning is expected and will not affect functionality.

View solution in original post

thahir
Path Finder

@Skeer-Jamf  you got any resolution for this issue

0 Karma

alek_cybersec
Explorer

And what about this: 


@_joe wrote:


I am also finding that "./splunk disable boot-start" does not correctly remove the /etc/init.d/splunk script


I've also noticed similar problem.

0 Karma

_joe
Communicator

This warning got added to "known issues" for 9.0.0

Date filed Issue number Description

2022-06-23SPL-226019Warning appears in the universal forwarder whenever any spl command is run: Warning: Attempting to revert the SPLUNK_HOME ownership Warning: Executing "chown -R splunk /opt/splunkforwarder". This warning is expected and will not affect functionality.

jfrench
Loves-to-Learn Lots

Try running btool when this "non-impacting" error occurs.  It doesn't run.

0 Karma

Skeer-Jamf
Path Finder

So a known issue and will not cause problems, that's great.  Any idea on a cause? Every time I call the splunk binary whether is starting/restarting the app or invoking btool.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...