I'm taking over Splunk admin duties from a co-worker that has left the company. We have a distributed environment setup of two heavy forwarders and four index servers all running Splunk Enterprise 7.3. I'm fleshing out our upgrade task list for moving to 8.1.2. What is the best way to manage the upgrade across all servers, and do I need to take any special steps to prevent loss of logs? I have the task for the install itself, I need a game plan for how to do it across the 6 servers. Will upgrading each server - one at a time, have it back up and running before I move to the next server - prevent log/data loss, or is it better practice to do all 6 servers at once (take them all down, upgrade them all, the bring them all back online). Looking at the compatibility matrix the I should be able to do all the indexers first and have the heavy forwarders still be compatible. Any advice on how to manage this is greatly appreciated.
Thanks for the information, Giuseppe. We do not have clustered indexers but we do have one primary search head. I've checked compatibility on the forwarders, so it appears one at a time, indexers then forwards is the way to go. Thank you!