Installation

Linux UF boot-start issues and non-impacting errors on 9.0.0

_joe
Communicator

Is anyone else running into boot-start/permissions issues with the 9.0.0 UF running on Linux using init.d scripts for bootstart?

Warning: Attempting to revert the SPLUNK_HOME ownership
Warning: Executing "chown -R splunk /opt/splunkforwarder"


I am also finding that "./splunk disable boot-start" does not correctly remove the /etc/init.d/splunk script and, contrary to documentation, splunk UF 9.0.0 uses systemd as default.
https://docs.splunk.com/Documentation/Splunk/9.0.0/Admin/ConfigureSplunktostartatboottime

Also systemd scripts seem to fail getting the permissions needed even when trying to enable-boot as root. A key error I am seeing is "Failed to create the unit file" when running the install. But it seems to be a total fail.

 

 

## When upgrading (from 8.2.5)
runuser -l splunk -c "/opt/splunkforwarder/bin/splunk stop"
tar -xzvf /tmp/splunkforwarder-9.0.0-6818ac46f2ec-Linux-x86_64.tgz -C /opt
chown -R splunk:splunk /opt/splunkforwarder/
runuser -l splunk -c "/opt/splunkforwarder/bin/splunk start --accept-license --answer-yes --no-prompt"
runuser -l splunk -c "/opt/splunkforwarder/bin/splunk status"
Warning: Attempting to revert the SPLUNK_HOME ownership
Warning: Executing "chown -R splunk /opt/splunkforwarder"
(NOTE: Seems to be non-impacting)

 

### When doing a new install
tar -xzvf /tmp/splunkforwarder-9.0.0-6818ac46f2ec-Linux-x86_64.tgz -C /opt
chown -R splunk:splunk /opt/splunkforwarder
[root]#  sudo -H -u splunk /opt/splunkforwarder/bin/splunk start --accept-license --answer-yes --no-prompt
Warning: Attempting to revert the SPLUNK_HOME ownership
Warning: Executing "chown -R splunk /opt/splunkforwarder"

This appears to be your first time running this version of Splunk.

IMPORTANT: Because an admin password was not provided, the admin user
 will not be created. You will have to set up an admin username/password
 later using user-seed.conf.
Creating unit file...
Current splunk is running as non root, which cannot operate systemd unit files.
Please create it manually by 'sudo splunk enable boot-start' later.
Failed to create the unit file. Please do it manually later.


Splunk> Now with more code!

sudo -H -u splunk /opt/splunkforwarder/bin/splunk status
Warning: Attempting to revert the SPLUNK_HOME ownership
Warning: Executing "chown -R splunk /opt/splunkforwarder"
splunkd is running (PID: 3132350).
splunk helpers are running (PIDs: 3132354).

 

# sudo -H -u splunk /opt/splunkforwarder/bin/splunk stop
Warning: Attempting to revert the SPLUNK_HOME ownership
Warning: Executing "chown -R splunk /opt/splunkforwarder"
Stopping splunkd...
Shutting down.  Please wait, as this may take a few minutes.
.                                                          [  OK  ]
Stopping splunk helpers...
                                                           [  OK  ]
Done.
# /opt/splunkforwarder/bin/splunk enable boot-start -user splunk
Systemd unit file installed by user at /etc/systemd/system/SplunkForwarder.service.
Configured as systemd managed service.


systemctl start SplunkForwarder.service
Job for SplunkForwarder.service failed because the control process exited with error code.
See "systemctl status SplunkForwarder.service" and "journalctl -xe" for details.

systemctl status SplunkForwarder.service
● SplunkForwarder.service - Systemd service file for Splunk, generated by 'splunk enable boot-start'
   Loaded: loaded (/etc/systemd/system/SplunkForwarder.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2022-06-21 12:58:55 UTC; 27s ago
  Process: 3141480 ExecStartPost=/bin/bash -c chown -R splunk:splunk /sys/fs/cgroup/memory/system.slice/SplunkForwarder.service (code=exited, status=0/SUCCES>
  Process: 3141478 ExecStartPost=/bin/bash -c chown -R splunk:splunk /sys/fs/cgroup/cpu/system.slice/SplunkForwarder.service (code=exited, status=0/SUCCESS)
  Process: 3141477 ExecStart=/opt/splunkforwarder/bin/splunk _internal_launch_under_systemd (code=exited, status=203/EXEC)
  Process: 3141475 ExecStartPre=/bin/bash -c chown -R splunk:splunk /opt/splunkforwarder (code=exited, status=0/SUCCESS)
 Main PID: 3141477 (code=exited, status=203/EXEC)

Jun 21 12:58:55 <host> systemd[1]: SplunkForwarder.service: Failed with result 'exit-code'.
Jun 21 12:58:55 <host> systemd[1]: Failed to start Systemd service file for Splunk, generated by 'splunk enable boot-start'.
Jun 21 12:58:55 <host> systemd[1]: SplunkForwarder.service: Service RestartSec=100ms expired, scheduling restart.
Jun 21 12:58:55 <host> systemd[1]: SplunkForwarder.service: Scheduled restart job, restart counter is at 5.
Jun 21 12:58:55 <host> systemd[1]: Stopped Systemd service file for Splunk, generated by 'splunk enable boot-start'.
Jun 21 12:58:55 <host> systemd[1]: SplunkForwarder.service: Start request repeated too quickly.
Jun 21 12:58:55 <host> systemd[1]: SplunkForwarder.service: Failed with result 'exit-code'.
Jun 21 12:58:55 <host> systemd[1]: Failed to start Systemd service file for Splunk, generated by 'splunk enable boot-start'.



Labels (2)
1 Solution

_joe
Communicator

This warning got added to "known issues" for 9.0.0

Date filed Issue number Description

2022-06-23SPL-226019Warning appears in the universal forwarder whenever any spl command is run: Warning: Attempting to revert the SPLUNK_HOME ownership Warning: Executing "chown -R splunk /opt/splunkforwarder". This warning is expected and will not affect functionality.

View solution in original post

thahir
Path Finder

@Skeer-Jamf  you got any resolution for this issue

0 Karma

alek_cybersec
Explorer

And what about this: 


@_joe wrote:


I am also finding that "./splunk disable boot-start" does not correctly remove the /etc/init.d/splunk script


I've also noticed similar problem.

0 Karma

_joe
Communicator

This warning got added to "known issues" for 9.0.0

Date filed Issue number Description

2022-06-23SPL-226019Warning appears in the universal forwarder whenever any spl command is run: Warning: Attempting to revert the SPLUNK_HOME ownership Warning: Executing "chown -R splunk /opt/splunkforwarder". This warning is expected and will not affect functionality.

jfrench
Loves-to-Learn Lots

Try running btool when this "non-impacting" error occurs.  It doesn't run.

0 Karma

Skeer-Jamf
Path Finder

So a known issue and will not cause problems, that's great.  Any idea on a cause? Every time I call the splunk binary whether is starting/restarting the app or invoking btool.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...