Deployment Architecture

Docker Start Breaks Cluster

nculpin
New Member

As we have no dev environment I have tried to learn Terraform and Ansible and build my own on Docker.

I now have 2 x Search heads in a cluster, 2 Indexers and an Indexer cluster master, 1 x heavy forwarder, 1 combined deployer/deployment server and a Universal forwarder.

Everything works fine and I can build the whole environment in a few minutes.

But if I stop the containers when I do a "docker start" the cluster configuration of the indexer cluster master and deployer are reset back to the default. 

This is the shclustering stanza of server.conf on the deployer when the environment is built:

[shclustering]
pass4SymmKey = $7$P6EHXzK5D7eS/B6970mBtVsoThkdIn27+xiyZdy2tkOAveg1O3o2rg==
shcluster_label = shcluster_label

And this is after the docker start:

[shclustering]
pass4SymmKey =
shcluster_label = shc_label

This is the clustering stanza from the indexer cluster master server.conf initially:

[clustering]
cluster_label = idxcluster_label
mode = master
search_factor = 1
pass4SymmKey = $7$WLLkzIXVZZmbtPcy1YDkhUNyKI1mzMMPz2Q0dTbivBHxFAokebPZose71eiT
replication_factor = 1

And this is after the docker start:

[clustering]
cluster_label = idxc_label
mode = master
search_factor = 3
pass4SymmKey =
replication_factor = 3

And in the logs for the indexer cluster master I can see this:

09-15-2020 12:56:34.296 +0000 INFO CMMaster - Creating CMMaster: ht=60.000 rf=3 sf=3 ct=60.000 st=60.000 rt=60.000 rct=60.000 rst=60.000 rrt=60.000 rmst=180.000 rmrt=180.000 icps=-1 sfrt=600.000 pe=1 im=1 is=0 mob=2 mor=5 mosr=5 pb=5 rep_port= pptr=10 fznb=10 Empty/Default cluster pass4symmkey=true allow Empty/Default cluster pass4symmkey=true rrt=restart dft=180 abt=600 sbs=1
09-15-2020 12:56:34.296 +0000 WARN CMMaster - pass4SymmKey setting in the clustering or general stanza of server.conf is set to empty or the default value. You must change it to a different value.

Note that server.conf is not totally replaced just the clustering stanzas. So that suggests ansible, but I can't find a anything that changes these stanzas. Note that the search heads are not changed and server.conf is unchanged after the "docker stop". 

 

 

 

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...