Splunk Enterprise

Upgrade to 9.3.2 appears to have broken my installation

zarchitect
New Member

Hi all, I was upgrading Splunk Enterprise from 9.0.x to 9.2.4 and then 9.3.2. When I try to restart the Splunk Service I get the following:

Failed to start Systemd service file for Splunk, generated by 'splunk enable boot-start'.
Unit Splunkd.service entered failed state.
Splunkd.service failed.
Splunkd.service holdoff time over, scheduling restart.
Stopped Systemd service file for Splunk, generated by 'splunk enable boot-start'.
start request repeated too quickly for Splunkd.service
Failed to start Systemd service file for Splunk, generated by 'splunk enable boot-start'.
Unit Splunkd.service entered failed state.
Splunkd.service failed.
 
I'll add from a Splunk standpoint I am a complete noob. I did some research on the upgrade process and followed the Splunk documentation. 
 
TIA!
Labels (1)
0 Karma

isoutamo
SplunkTrust
SplunkTrust
Was this issue wit 9.2.4 or after that when you are starting it wit 9.3.2?
Which Linux os distro and version you have and are those same as earlier?
0 Karma

zarchitect
New Member

Starting with 9.3.2. It's running on Amazon Linux 2.

0 Karma

isoutamo
SplunkTrust
SplunkTrust
In which user you are running it and how (exactly) you did upgrade?
Is this all in one or distributed environment?
There was no error when you start it with 9.2.4?
0 Karma

zarchitect
New Member

All-in-one environment. The user account on the machine I used was ec2-user. I assume, but am not sure if that was the user used to do the original install. 

Honestly, I didn't try to start the instance after the 9.2.4 upgrade. I was on 9.0. Did the applied the 9.2.4 upgrade and then immediately applied the 9.3.2 upgrade. 

I user the tgz upgrade file and followed the 9.3.2 upgrade documentation.

Again, total noob here. 

0 Karma

isoutamo
SplunkTrust
SplunkTrust
Unless you start splunk with all those mid versions it didn’t do those conversations etc actions which are needed before next update. Now you have done direct update from 9.0.x to 9.3.2 and this is not supported way.
Usually splunk has installed as root, but it should run as splunk (or other non root) user.
Have you look what logs said especially migration.log and splunkd.log?
0 Karma
Get Updates on the Splunk Community!

Now Available: Cisco Talos Threat Intelligence Integrations for Splunk Security Cloud ...

At .conf24, we shared that we were in the process of integrating Cisco Talos threat intelligence into Splunk ...

Preparing your Splunk Environment for OpenSSL3

The Splunk platform will transition to OpenSSL version 3 in a future release. Actions are required to prepare ...

Easily Improve Agent Saturation with the Splunk Add-on for OpenTelemetry Collector

Agent Saturation What and Whys In application performance monitoring, saturation is defined as the total load ...