When the deployer sends a restart with /opt/splunk/bin/splunk restart, that changes the value I get with systemctl status Splunk.
It sets it from active (running) to inactive (dead).
We check this value to control scripts and so forth.
How can we either get the deployer to send restarts with systemctl restart Splunk or get the systemctl to properly acknowledge that the system is up after an /opt/splunk/bin/splunk restart?
Summary of the issue:
Splunk 6.0.0 - Splunk 7.2.1 defaults to using init.d when enabling boot start
Splunk 7.2.2 - Splunk 7.2.9 defaults to using systemd when enabling boot start
Splunk 7.3.0 - Splunk 8.x defaults to using init.d when enabling boot start
systemd defaults to prompting for root credentials upon stop/start/restart of Splunk
Here is a simple fix if you have encountered this issue and prefer to use the traditional init.d scripts vs systemd.
Splunk Enterprise/Heavy Forwarder example (note: replace the splunk user below with the account you run splunk as):
sudo /opt/splunk/bin/splunk disable boot-start
sudo /opt/splunk/bin/splunk enable boot-start -user splunk -systemd-managed 0
Splunk Universal Forwarder example (note: replace the splunk user below with the account you run splunk as):
sudo /opt/splunkforwarder/bin/splunk disable boot-start
sudo /opt/splunkforwarder/bin/splunk enable boot-start -user splunk -systemd-managed 0