Deployment Architecture

Can you verify my plans for a search head cluster configuration?

allan_newton
Path Finder

Hi All,

I'm trying to create a sh cluster, here are the sequential things that I have. Please correct me.

On the deployer
[shclustering]
pass4SymmKey = shc@cluster
shcluster_label = sh_cluster

restart the deployer

On all the search heads except deployer
splunk init shcluster-config -auth admin:changeme -mgmt_uri https://respective_sh_ip:mgmt_port -replication_port rep_port -replication_factor 2 -conf_deploy_fetch_url https://deployer_ip:mgmt_port -secret shc@cluster -shcluster_label sh_cluster

restart the search heads after the configuration.

Only on one search head and not deployer
splunk bootstrap shcluster-captain -servers_list "https://sh1_ip:mgmt_port,https://sh2_ip:mgmt_port,https://sh3_ip:mgmt_port" -auth admin:admin

Push configurations from deployer to sh member.

create a app and moved it to etc/shcluster on deployer, then

splunk apply shcluster-bundle -target https://sh1_ip:mgmt_port

Thanks,
Allan

0 Karma
1 Solution

kunalmao
Communicator

Everything looks fine, you will have to incorporate comment made by jdosch1 if you haven't , apart from that i would suggest you test your search head cluster by taking the captain down and then running ./splunk shcluster-status to see if election of new captain is being done smoothly before applying changes from deployer.

Also add the below stanza in server.conf of all the members to ensure that your search head captain is not loaded with scheduled searches and is performing optimally

[shclustering]
captain_is_adhoc_searchhead = true

View solution in original post

0 Karma

koshyk
Super Champion

better to put the above CLI command parameters (eg replication_factor etc.) into .conf files, so that you can version control and automate

0 Karma

kunalmao
Communicator

Everything looks fine, you will have to incorporate comment made by jdosch1 if you haven't , apart from that i would suggest you test your search head cluster by taking the captain down and then running ./splunk shcluster-status to see if election of new captain is being done smoothly before applying changes from deployer.

Also add the below stanza in server.conf of all the members to ensure that your search head captain is not loaded with scheduled searches and is performing optimally

[shclustering]
captain_is_adhoc_searchhead = true

View solution in original post

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Looks correct to me.

The one gotcha with SHC is that all local folders will be merged with default on the deployer

During the staging process that occurs prior to pushing the configuration bundle, the deployer copies the configuration bundle to a staging area on its file system, where it merges all settings from files in /shcluster/apps/<appname>/local into corresponding files in /shcluster/apps/<appname>/default. The deployer then pushes only the merged default files.

http://docs.splunk.com/Documentation/Splunk/7.0.0/DistSearch/PropagateSHCconfigurationchanges

0 Karma

jdosch1
Engager

Looks good.
have you already configured the SH to contact the Cluster Master ? If not you need to run the CLI below
./splunk edit cluster-config -mode searchhead -master_uri https://clustermaster.dev.com:8089 -secret

richgalloway
SplunkTrust
SplunkTrust

Looks OK to me.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.