I want to upgrade my Splunk from 4.0.9 to 4.1.6 My system is a RHEL box.
Queries regarding this:
From what I have read, I simply run the new Splunk (RPM file) & it will overwrite the files & try & use existing configuration , with manual prompts for stuff Splunk is unsure.
Also I have external SAN storage mapped for splunk at /opt/splunkdatabase
Since this is referred via configuration /opt/splunk/etc/splunk-launch.conf, I presume no issues in accessing existing historic logs in this for my current configured indexes?
I am planning an upgrade from 4.12 to 4.17 very soon with a similar configuration (OS, SAN usage) I reviewed the process with tech support. Since splunk is a self contained aplication, Most of your points should be covered ( your config, & Murphy's law permitting) I have done many upgrades successfully. Here are ome guidelines that may be useful and hopefully practical.
Basically my Plan is 1) Create a sound backup of your splunk directory and databases, test restore too. 2) Test the installation on a simlar system. May not be able to replicate exact config, and san instance. Never hurts to try it once. I have upgraded a Splunk light Forwarder from 4.14 to 4.17 using the rpm -Uvh option with out any significant issues. You could also test your rollback plan if the upgrade does not meet your criteria. 3) Housekeeping: Document Searches,Review sources and sourcetypes (Remove obsolete items) , license info, applications and or dashboards. LDAP configuration and any other proprietary info. 4) Stop the splunk application before you upgrade, no need to worry about the LWF's. (Beware of dormant LWF's that come back online, they try to archive all of the data that may have been missed while dormant) 5) Test test test - Scheduled alerts, reporting, searches, user authentication, your backups. 6) Also beware of system updates, I have had to rebuild my HBA SAN interfaces (QLogic) utils after a OS kernel upgrade. 7) Benchmark your LWF's datathrough put, sourcetype input , the all time real time serach is a great place to begin with the _internal index and metrics logs.
I have the great fortune of being in a ITIL environment, so I get to write a RFC and have it reviewed by a CAB advisoy before I upgrade our prod splunk server. I will keep you updated if you would like?
Remember that you always have an option to download the old version of Splunk and test the upgrade on another lab system. This would allow you to see what exact prompts appear during an upgrade.
Also, @jmulcaster_splunk just posted an order-of-operations diagram with links to relevant documentation to help with upgrade planning. Check it out and let us know if you find it helpful. What's the order of operations for upgrading Splunk Enterprise?