Is it possible to stop the forwarder instead of restarting it when deploying a new app?

[yhSapSplunk]

Linux 4.2.0-27-generic #32~14.04.1-Ubuntu

in /var/log/syslog
Nov 24 04:10:25 kernel: [6690966.699401] init: splunk main process ended, respawning
Nov 24 04:10:26 kernel: [6690967.095850] init: splunk main process (27149) terminated with status 1

We have a upstart job script created for Splunk and it seems working fine to start/stop the Splunk process. But when a new app is deployed to the forwarder via the deployment server and the Splunk process on the forwarder is restarted, it seems that the upstart job manager lost the track of the process and keep spawning (of course without success).

Question 1: is there a way to just stop the forwarder instead of restarting it while an app is deployed? If so, the upstart job manager should be able to detect the process is down, start it and keep track of it.

Question 2: is there a way to modify our upstart job to pick up the new process restarted and keep track of it?

[yhSapSplunk]

restartSplunkd has true or false value.
What I'm looking for is an stop action, not restart. Anytime a change is made to an app, I want the forwarders pull the change and stop the splunk process. The upstart manager should be able to detect that and hence start the process automatically and manages it.

[ddrillic]

The administration guide for the certification shows the UI for the Post Deployment Behavior Setting. It lets the user choose which action to take after the app deployment.

[ryanhast]

The 'restartifNeeded' set to true in the serverclass.conf may take care of the restart always when an app is deployed to the forwarder.

restartIfNeeded = true | false
* This is only valid on forwarders that are newer than 6.4.
* If true and issueReload is also true, then when an updated app is delpoyed
to the client, that client will try to reload that app. If it fails, it will
then restart.
* defaults to false
https://docs.splunk.com/Documentation/Splunk/6.5.1/Admin/Serverclassconf

Here is post on answers that may point you in the direction with working with 'upstart' and splunkd.
https://answers.splunk.com/answers/22511/integrating-splunk-with-upstart-ubuntu-debian.html

[ddrillic]

A similar qustion at Deployment Server - reload configs without restarting splunk

We also use /opt/splunk/bin/./splunk reload deploy-server after making changes to a deployment app or/and to the serverclass.conf file and it triggers a restart of the forwarder that got the changes.

Very interesting question.

Based on splunk reload deploy-server not updating bundles

it seems that the reload command might be modified -

[ddrillic]

restartSplunkd is available in the serverclass stanza...

[yhSapSplunk]

restartSplunkd controls restart action (either restart or not after a forwarder pulls new changes). What I'm looking for is a stop action. That way, the upstart manager will detect that the process is down so it can start it and keep managing it.