What's the best way to separate "cluster master" and " deployer"

[mlevsh]

We are running Splunk Enterprise 7.0.1 with 4 servers on search head cluster
and 8 indexers on indexer cluster, multi-site

We have one server combining two functions:

1) "cluster master" to deploy apps and manage replication within an indexer cluster
2) "deployer" - to deploy apps to a search head cluster

After recent upgrade from 6.5.3 to 7.0.1 we realized, that it would be beneficial to have a designated server for each of the functions above.

Can you please recommend the best way to accomplish the separation of two functions to two designated servers?

[ddrillic]

In our installations we kept the deployment and deployer servers together on the same physical server. Not sure if there was rhyme or reason for that.

[mlevsh]

@ddrillic we have deployer and cluster master sharing the same server. During the recent Splunk upgrade we needed to upgrade search head cluster before upgrading indexers cluster , but couldn't. Because a deployer had to be upgraded and restarted but it is also a cluster master , which we couldn't upgrade yet.

[ddrillic]

Really interesting question. Looking at our upgrade layout -

enable_maintenance_mode

shutdown master
shutdown indexers
shutdown seach heads
shutdown batch heads
shutdown deploy/license server
check all shutdown

backup /opt/splunk to /SplunkConfigBackup

upgrade master
start master
put master in maintenance mode

upgrade indexers
upgrade sheach heads
upgrade batch heads
upgrade deploy/license server

start indexers
start search heads
start batch heads
start deploy/license server

check all status

disable maintenance mode

This layout shows clearly that the cluster master has to be by itself.

Thank you @mlevsh.

[Steve_G_ · Accepted solution]

It would be simpler to move the deployer to a new Splunk instance and leave the cluster master where it is.

To configure a new deployer, see this subtopic: http://docs.splunk.com/Documentation/Splunk/7.0.2/DistSearch/PropagateSHCconfigurationchanges#Config...

You'll also need to move the $SPLUNK_HOME/etc/shcluster/ directory from the old deployer to the new one.

You might be able to colocate the deployer with some other management function, such as the license master or the monitoring console. For information on colocation possibilities for the deployer, see: http://docs.splunk.com/Documentation/Splunk/7.0.1/DistSearch/SHCsystemrequirements#Deployer_requirem...

[mlevsh]

@Steve G. [Splunk] , thank you for your advice!
What are pros and cons to move a deployer versus moving a cluster master?

Thanks!

[Steve_G_]

The role of the master node in an indexer cluster is much more significant than the role of the deployer in a search head cluster. In addition to managing app and conf file deployment, like the deployer, it manages the relationships between the various nodes, and so on.

So replacing a master node requires a much more careful approach than replacing a deployer. But you can do it that way, if you want to. See http://docs.splunk.com/Documentation/Splunk/7.0.2/Indexer/Handlemasternodefailure

[Joao_HCL]

@mlevsh right now I face the same issue.
Need to upgrade both, SH and IDX clustered environments, and have the Cluster Master, License Master and Deployer configured in the same instance.
Due to Splunk upgrade plan the CM is the first to upgrade and to start, the Deployer should only be upgraded after upgrading the according SH cluster members.
How did you proceed here?