Splunk Enterprise

Multi Site Index Cluster Deployment Suggestions

akhan18
New Member

Hello,

We are trying to deploy a multi-site cluster for high availability and disaster recovery process.  I looked at the Splunk valid architecture document and decided to move forward with M2/M12 deployment model with 2 indexers on each side instead of 3. I wanted to run my rational through some experts

Current Architecture 

  • Search Head 
  • 2 Indexers Configured for Distributed Search (12 CPU, 16GB RAM and 2TB) 
  • 1 Heavy Forwarder that receives logs from a SysLog server 
  • 70GB Daily License Usage (Plan on having 2 replicas and 1 searchable copy) 
  • Version 8.1.1
  • Ubuntu 18.04 latest patched
  • No Deployment Server
  • License Master and Monitoring Console

    I had a few questions 

    (i) What are the hardware requirements for the Splunk Indexers in a Clustered environment? 
    Note: Currently both Indexers are running on 12 CPU, 16GB RAM and 2TB Storage in VSphere. 
    I bumped that up to what Splunk recommends here along with 250GB Primary and10TB Secondary Disk 

    Mid-range indexer specification

    • An x86 64-bit chip architecture.
    • 24 physical CPU cores, or 48 vCPU at 2GHz or greater speed per core.
    • 64GB RAM.
       
      (ii) What are the hardware requirements for the Splunk Search Head? I am not planning on deploying a search head cluster and only a few users query Splunk. Is this okay?
      An x86 64-bit chip architecture.
      12 physical CPU cores. 
      32GB RAM.
      250 GB Primary Disk 

      (iii) I have Cluster Master that is on the Primary side that will manage the Index Cluster. I made this a separate server with decent specs. What are the recommended specs for a Cluster Master. I just hardened the server and add the Splunk Installer to it. Is there anything else I need to do before I configure the cluster?

      (iv) The existing indexers have disks that were partitioned using LVM so I can easily extend the 2TB to 10TB. How should I go about this with Splunk Indexers?  Any gotchas I have to look out for before lvextend -L +8T /dev/mapper ?

      (v) I have Universal forwarder and one heavy forwarder. Would I need to enable index discovery on both universal forwarder and the heavy forwarder in outputs.conf?

      (vi) I have two indexers on each side now that match the same specs. Can I join the cluster before the indexer discovery change or after? Are there any gotchas I should look out for before joining the cluster? I have backups and replication between both sites. I expect the traffic to be much higher between sites when replication is turned on. 

      (vii) We also have a KVStore, do I need to do something special for it when an Index Cluster is deployed?

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!