What to do first: Cluster Indexers, then Upgrade Linux, or upgrade Linux, then cluster indexers?

[gozulin]

We have 2 indexers (one site) that are running on Redhat 6.2 that we want to upgrade to 6.7 for security reasons.

We also want to cluster them.

Should we cluster first, then upgrade the OS on one indexer at a time, or upgrade them, then cluster them?

Which is less risky?

Currently, all our forwarders are configured so they can send to both Indexers, like so:

[tcpout]
defaultGroup = default-autolb-group

[tcpout:default-autolb-group]

server = 10.1.1.2:9997 , 10.1.1.3:9997
useACK=true

[tcpout-server://10.1.1.2:9997]
[tcpout-server://10.1.1.3:9997]

So either way, stopping one indexer should make the forwarders send to the other indexers, cluster or no cluster, am I right?

The only difference I can see is if we cluster first, search results would not be impacted during our maintenance window.

So, what say thee?

[Yasaswy]

Hi gozulin,
stopping one indexer should make the forwarders send to the other indexers, cluster or no cluster, am I right? yes
As per clustering, you would need a minimum of 3 indexers. Just by the info provided above, I don't think you can cluster above 2 mentioned indexers for HA.

So you cannot avoid disruption of service (searches would have incomplete data) during the upgrade... but as you mentioned above you are not loosing any inbound data.

If you have a new server available for indexer ... then yes cluster first and upgrade one server at a time so you have no service disruption. If service disruption in not a big deal... it's cleaner/easier to upgrade first and cluster 🙂

[gozulin]

how is it cleaner/easier to upgrade first and then cluster?

[Yasaswy]

When you cluster splunk you typically will have more things to consider than you would in the current state. Clustering itself will require some amount of planning (even with just 2 peer nodes and a cluster master). So assuming service interruption is acceptable ... to me the easier option (relatively speaking) appears to be finishing off the upgrade first and plan and do cluster deployment later.... So I am coming from the perspective that you need to pick one of these 2 choices immediately.

[somesoni2]

For indexer cluster, there is no minimum node requirement (2 will do as well). The number of nodes required is depending upon the replicationFactor (no of node in indexer cluster=replication factor).

[Yasaswy]

True. Thanks for correcting 🙂