Deployment Architecture

Adding a new indexer to the indexer cluster

Builder

Can I please know the process of adding a new indexer to the indexer cluster ?

Should the cluster be kept in maintenance mode while adding the new indexer ?

Should the secret key be added in general stanza or the clustering stanza ?

What is the difference between splunk.secret and pass4SymmKey ?

1 Solution

SplunkTrust
SplunkTrust

The process for adding an indexer to a cluster is documented at https://docs.splunk.com/Documentation/Splunk/8.0.1/Indexer/Addclusterpeer . The steps apply to earlier versions of Splunk, not just 8.0.1.

---
If this reply helps you, an upvote would be appreciated.

View solution in original post

SplunkTrust
SplunkTrust

The process for adding an indexer to a cluster is documented at https://docs.splunk.com/Documentation/Splunk/8.0.1/Indexer/Addclusterpeer . The steps apply to earlier versions of Splunk, not just 8.0.1.

---
If this reply helps you, an upvote would be appreciated.

View solution in original post

Splunk Employee
Splunk Employee

I'm in a similar situation but the twist is that I need to remove an existing peer from an old cluster and take the peer (and its existing data) to join a new cluster / CM. The peer and its data is not to be modified other than what is required to connect to the new cluster master.
Is this the approach to take ? - take peer offline permanently in old cluster, remove peer from old CM, modify server.conf on peer with new master_uri and pass4SymmKey, restart peer to join new cluster.
Any recommendations are welcome. Thanks in advance!

0 Karma

Champion

After adding a peer to the indexer cluster, we may need to do "Data Rebalance" across the indexer cluster(because the new indexer may get new data, but the "share" of the old data may not be available on new indexer)

Good Reading about Data Rebalancing on Splunk Indexer Cluster -

https://conf.splunk.com/files/2016/slides/rebalancing-data-across-an-indexer-cluster.pdf

https://conf.splunk.com/files/2016/slides/lets-rebalance-data-across-an-indexer-cluster-in-15-minute...

0 Karma

New Member

For anybody that is rolling a indexer that's manged by a deployment server. The deploymentclient.conf will need to be removed and any apps that were deployed by the ds will need to removed.

0 Karma

SplunkTrust
SplunkTrust

1) High level steps would be these:

1) enter maintenance mode on the cluster master...
2) up the search factor/replication factor (if desired)...
3) enable indexer clustering on the new indexer and join the indexer to the master (peer node configuration): http://docs.splunk.com/Documentation/Splunk/7.0.1/Indexer/Enablethepeernodes
4) Ensure all indexes are recreated on the new indexer
5) Data rebalance, if desired
6) Bring master out of maintenance mode
7) Push out new outputs.conf to forwarders with 3rd indexer info as well distsearch.conf to SH (if using that method)

2) Yes
3) Security key (same as the secret key used while setting up clustering) should setup in clustering stanza in server.conf (http://docs.splunk.com/Documentation/Splunk/7.0.1/Indexer/Configurepeerswithserverconf)
4) The splunk.secret key is for encrypting credential (see this). The pass4SymmKey is used to secure inter-splunk communication on clustering (see this)

Builder

Thanks for the answer @somesoni2 ... If we keep the pass4SymmKey in [clustering] stanza then what should we keep in the [general] stanza of the server.conf ?

Is the pass4SymmKey value present in the [general] stanza same in all the instances irrespective of indexer or search head ?