Search Head Cluster - Hardware considerations

[leefernan]

Hello Everyone,

I have an environment which has an index cluster and three search heads that are currently looking for data in this cluster. 

I want to create a SH cluster with this three search heads, but the hardware specifications between  them are different:

-SH1 40 Cores 128GB Ram, (Chosen as captain)

-SH2 24 Cores 64GB Ram, (Member)

-SH3 24 Cores 64GB Ram, (Member)

The Splunk documentation specifies that "Use identical specifications for all members (bare metal or VM)" 

What would be the impact or implications to deploy a search cluster with this servers different  in Hardware Specifications? 

The captain will use only just 24 cores and  64gb ram as the other cluster members? 

Or the captain will assume every server has the same hardware capabilities as him?  As the following text suggest: 

"Splunk recommends that you use homogeneous machines with identical hardware specifications for all cluster members. The reason is that the  cluster captain assigns scheduled jobs to members based on their current job loads. When it does this, it does not have insight into the actual processing power of each member's machine. Instead, it assumes that each machine is provisioned equally."

I will appreciate your knowledge, thoughts and recommendations. 

Thanks in advance. 

[richgalloway · Accepted solution]

The SHC captain assumes all nodes are the same as itself.  That means it could give each member 46 searches (# CPUs + 6) when they can support only 30.  Search performance likely will suffer.

BTW, the cluster typically chooses its own captain, which means SH2 or SH3 could become captain and assume SH1 only supports 30 searches.  It that case, some resources are wasted.

[leefernan]

 I suspected that. It's nice to have a confirmation. Thanks a lot!