Will I be able to upgrade indexers in a cluster without any downtime?

[daniel333]

All,

A bit concern for us lately is Splunk downtime. Search head clustering has been helpful, so now we're looking at the indexing tier..

Per my reading here:
http://docs.splunk.com/Documentation/Splunk/6.5.1/Indexer/Takeapeeroffline

Seems I can upgrade node by node without downtime as along as my search factor is high enough? Am I reading this correctly?

[lguinn2]

Yes, you can upgrade without downtime. You will want to put the entire cluster into maintenance mode, then upgrade and restart each indexer in turn. The cluster master will redirect searches to the currently operating indexers. After all upgrades are complete, you can take the cluster out of maintenance mode. At that point, it will catch up on any replication that was missed during the upgrade.

If you have a search factor of 2, you can take down 1 indexer at a time for upgrades, and search will continue to work.

[jjarrell]

Iguinn, this appears to disagree with the documentation here:

http://docs.splunk.com/Documentation/Splunk/6.6.3/Indexer/Upgradeacluster

"Caution: When upgrading a 6.x single-site indexer cluster, such as 6.2, to a later 6.x cluster, such as 6.3 or 6.4, you must take down and upgrade all peer nodes as a single operation. You cannot perform a rolling, online upgrade of the peer nodes."

Is this a conflict, or am I missing something obvious?

[j0shrice]

Lguinn I agree with you but I also think it's worth noting that if there was any legacy data from before the cluster was built on any indexers currently (older data existing on any standalone splunk indexers originally), then that data will not be available to the search heads when that particular indexer is down - unless the old data was copied to be replicated. Probably not something Daniel will have to worry about, but it's good to thoroughly understand that index clustering when a cluster is first set up only replicates new data coming into Splunk.

[lguinn2]

Excellent point!