Splunk is generally upgraded from the SH tier down. The ES staging instance implies you’re running ES on the SHC.
Verify the version of ES you’re running supports installation on 6.4.4. If not, upgrade it to a version that does.
Upgrade the CM first. I added this step to clarify the order-of-ops process defined in the docs.
Upgrade all SHC nodes, and upgrade the deployer following the documented steps.
Place the CM into maintenance mode and upgrade the index cluster. All clustered indexers should be taken down for the upgrade, as at this time upgrading some indexers while leaving others running is only supported for maintenance releases (e.g. 6.4.1 to 6.4.1.1.)
Disable maintenance mode on the CM. Note: you must finish the upgrade on all indexer nodes before disabling maintenance mode.
Upgrade supporting nodes such as the deployment server.
Upgrade any staging instances.
I have linux hosts so I’ll use a distributed shell to apply the same command across multiple nodes:
1. Grab a backup of the core Splunk configurations. You can copy the folder on each node to a new folder (ugh! And it assumes that any index buckets are NOT under %splunk_home%,) or just run diags to keep a copy.
2. Follow the upgrade instruction noted above per-tier: Untar the latest release on top of the old installation. Start services, and check for errors. Bring the tiers back up in the order suggested in the docs. NOTE: Due to special restrictions on clustered nodes, read the doc links above carefully as there are order-of-ops nuances. Your clusters will be down for a time, so make sure you understand what data sources in your environment will show gaps. For example, data polled for indexing using a script may miss an interval, any UDP data sources going to a Splunk instance you stop services on will not be indexed, etc.
3. Check that the basic services are working: LDAP logins, key apps, scripted inputs, critical data source checks, forwarder management has check-ins, license server is shows all’s well, CM show’s all’s well, etc.
4. This is a brilliant time to document the instance-specific details of your upgrade process. Include the validation checks you ran in your docs.
... View more