What's the order of operations for upgrading Splunk Enterprise?

[jmulcaster_splu]

I'm planning an upgrade to the latest version of Splunk Enterprise. What is the high-level order of operations? Is there an intermediate step required if I'm on Splunk 6.5 or earlier? Where do forwarders and premium apps fit in? What docs do I need to refer to help me plan and execute my upgrade?

[jmulcaster_splu · Accepted solution]

Glad you asked! We've created a high-level process road map for upgrading Splunk Enterprise, forwarders and apps. This process works for all Splunk Validated Architectures - just skip the components that aren't relevant to your deployment.

This diagram is for planning purposes only. It is not a comprehensive upgrade plan, and does not include technical details for upgrading. Please refer to the linked documentation for the version of Splunk you're upgrading to before you proceed with an upgrade.

Remember these operational best practices for upgrading:

• Create a detailed upgrade plan
• Develop a timeline to prepare for upgrade, and a schedule for your live upgrade window
• Identify everyone in your org who is affected by the upgrade
• Communicate your timeline to everyone who's affected by the upgrade. For communication plan best practices, see Communication best practices for a Splunk deployment in the Splunk Success Framework Handbook.

(click to enlarge and download as pdf)

---

What's your experience? We'd like to hear from you. We'll be updating this graphic as we gather more input.

[jmulcaster_splu]

Update 1/24/20: I verified that the upgrade order-of-ops graphic is up-to-date for upgrading to Splunk Enterprise 8.0+, and gave it a little refresh. I also streamlined the doc links on the right.

[jmulcaster_splu]

Great news! I've just updated this post with links to David Paper's new posts for what to monitor before, during, and after an upgrade:

• How do I benchmark system health before a Splunk Enterprise upgrade?
• How do I monitor system health during a Splunk Enterprise upgrade?
• What do I validate after I upgrade Splunk Enterprise to confirm the upgrade was successful?

These links are slotted into the activities they relate to in the diagram.

[jmulcaster_splu]

I just posted an update to the upgrade order-of-operations graphic to include a check for the support status of Splunk-built apps and add-ons before upgrading.

See End of Availability: Splunk-Built Apps and Add-Ons.

[DavidHourani]

Is there any guide for troubleshootingfailed upgrades ? And also a best practice for rollbackthat can be included ?

[sloshburch]

@DavidHourani - my understanding is that rollback (or downgrade from a later release to an earlier) is not supported. Therefore, we'll be likely avoiding that topic as we don't want to encourage folks to compromise their platform in a way that support will not be able to help.

[jmulcaster_splu]

Hi, @DavidHourani, for now, you can refer to the Splunk Enterprise troubleshooting overview, General troubleshooting issues for distributed search, and Introduction for troubleshooting Splunk Enterprise on Splunk Docs. We'll be posting more upgrade best practices to the validated_best-practice and upgrade tags here on Answers, too, so stay tuned!

[sloshburch]

Sorry for the broken image and link folks! All fixed now.

[bgronvall_splun]

I would add the following:

1. Identify the type of architecture your splunk environment contains(standalone, distributed, indexer clustering, search head clustering, premium apps(ITSI/ES/etc)).
2. Read the Known Issues for the version you are planning to upgrade to.
3. Read New Features for the version you are planning to upgrade to.
4. Confirm your apps are compatible to the version you upgrade to.
5. If possible test the upgrade in a replicated "dev" environment to ensure functionality of all your apps/add-ons/etc.
6. Review functionality changes for new versions + outdated configs.
7. Backup your data.

Standalone:
https://docs.splunk.com/Documentation/Splunk/7.2.6/Installation/HowtoupgradeSplunk

Distributed:
https://docs.splunk.com/Documentation/Splunk/7.2.6/Installation/UpgradeyourdistributedSplunkEnterpri...

Indexer Cluster:
https://docs.splunk.com/Documentation/Splunk/7.2.6/Indexer/Upgradeacluster

Search Head Cluster:
https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/UpgradeaSHC

[jmulcaster_splu]

Thanks for the input, msykes & bgronvall!

The roadmap does point to the release notes, where the known issues and new features are listed, and to the READ THIS FIRST upgrade considerations topic in docs, which covers functionality changes for new versions and outdated configs. I'll call those specific checks out in the boxes, too.

Let us know if you think of more things to enhance this general roadmap, or something we can add to a future post.

[msykes_splunk]

Would be good to update post to incorporate @bgronvall_splunk's bullets 2, 3, & 6 in the Prepare phase. I think the rest is already there.

[sloshburch]

Thanks @bgronvall_splunk! Me thinks you posted this while the image was broken. Now that it's up there, do you feel it captures your details or still missing things?

[bgronvall_splun]

yeah the image wasn't there...