Single Cluster to Multisite Cluster conversion

[mudragada]

Hi,

I have a single site cluster right now with the below configuration.
1. One License Server - also deployment server (001)
2. One Cluster Master (002)
3. Two Indexers (003, 004)
4. Two Heavy Forwarders (005, 006)
5. Two Search Servers (007, 008)
All the above servers are now opened for TCP 8089 between each other. There are about 100 Splunk Forwarders forwarding data to this 8 server cluster in Site1.
Here is the output of cluster-config on license master.
config
access_logging_for_heartbeats:1
cxn_timeout:60
disabled:0
forwarderdata_rcv_port:?
forwarderdata_use_ssl:0
guid:xxxxxxxx
heartbeat_period:4222054400
heartbeat_timeout:60
master_uri:https://002:8089
max_auto_service_interval:30
max_peer_build_load:5
max_peer_rep_load:5
max_peer_sum_rep_load:5
mode:searchhead
multisite:false
notify_scan_period:10
percent_peers_to_restart:10
ping_flag:1
quiet_period:60
rcv_timeout:60
rep_cxn_timeout:60
rep_max_rcv_timeout:600
rep_max_send_timeout:600
rep_rcv_timeout:60
rep_send_timeout:60
replication_factor:3
replication_use_ssl:0
restart_timeout:60
search_factor:2
search_files_retry_timeout:600
secret:*******
send_timeout:60
service_interval:1
site:default*

Now the plan is to setup another site - that'll also have about another 50 Splunk Forwarder that need to forward logs to the same set of indices. I have the below questions now.
1. I still need the 8 servers in the new Site2 - but the deployment server will be acting as a license slave. is that correct? I've already installed Splunk Enterprise same version (6.5.2) on about 7 servers and didn't do any configurations further.
2. Assuming servers 101 to 108 will be available in Site2 and the above configuration - what commands should I be executing to configure multisite clustering on all these 16 servers?
3. What values of search_factor and replication_factor should I consider?
4. For these servers to form a multisite cluster opening firewall ports between Site1 and Site2 - for splunk management and replication? How do I configure these replication ports?
5. I'd like to setup replication first and make sure that the logs of Site1 are searchable in Site2 search servers. Is that the right approach?
6. The Site2 Deployment Server is going to be a backup for Site1 Deployment Server or is going to be a new Deployment Server?

The end goal is to make the 4 search servers (2 from Site1 and 2 from Site2) be able to serve the same data - with 100 forwarders to Site1 and 50 forwarders to Site2.

[skalliger · Accepted solution]

Hi,

I think many of your questions will be answered here: http://docs.splunk.com/Documentation/Splunk/6.6.0/Indexer/Migratetomultisite

1. Make sure your Splunk versions meet the recommendation.
2. Prepare the servers (commands)
3. Your buckets will not be migrated. Only data that will be written after the multi-site cluster is created, will be searchable across both sites.
4. Everything besides the license master is a license slave. Because you can only have one license master, your deployer (or deployment server) will be a license slave.
5. Search and replication factor depend on your needs. Replication port is also listed on the website (9887).

Any further questions?

Skalli