Splunk Update from 7.2.1 to 8.1.4 not working

[daymar_23]

Hello! 

I was trying to update my splunk version from 7.2.1 to 8.1.4, I made all the steps required here: 

https://docs.splunk.com/Documentation/Splunk/8.1.2/Installation/AboutupgradingREADTHISFIRST?utm_medi...

and Here

https://docs.splunk.com/Documentation/Splunk/8.1.2/Installation/UpgradeonWindows

But after run the Upgrade Splunk Enterprise using the GUI installer my Splunk is broken. 

In the splunk.log file there is the following: 

 Finished generating checksums for datamodel and report acceleration bucket summaries for all indexes.
[App Key Value Store migration] Checking if migration is needed. Upgrade type 1. This can take up to 600seconds.
WARN: [App Key Value Store migration] Service(36) terminated before the service availability check could complete. Exit code 14, waited for 2 seconds.
App Key Value Store migration failed, check the migration log for details. After you have addressed the cause of the service failure, run the migration again, otherwise App Key Value Store won’t function.
[App Key Value Store migration] Checking if migration is needed. Upgrade type 2. This can take up to 600seconds.
WARN: [App Key Value Store migration] Service(36) terminated before the service availability check could complete. Exit code 14, waited for 0 seconds.
App Key Value Store migration failed, check the migration log for details. After you have addressed the cause of the service failure, run the migration again, otherwise App Key Value Store won’t function.
[DFS] Performing migration.
[DFS] Finished migration.

But I review the migration.log and there are not more information: 

Generating checksums for datamodel and report acceleration bucket summaries for all indexes.
If you have defined many indexes and summaries, summary checksum generation may take a long time.
Processed 3 out of 36 configured indexes.
Processed 6 out of 36 configured indexes.
Processed 9 out of 36 configured indexes.
Processed 12 out of 36 configured indexes.
Processed 15 out of 36 configured indexes.
Processed 18 out of 36 configured indexes.
Processed 21 out of 36 configured indexes.
Processed 24 out of 36 configured indexes.
Processed 27 out of 36 configured indexes.
Processed 30 out of 36 configured indexes.
Processed 33 out of 36 configured indexes.
Processed 36 out of 36 configured indexes.
Finished generating checksums for datamodel and report acceleration bucket summaries for all indexes.
[App Key Value Store migration] Checking if migration is needed. Upgrade type 1. This can take up to 600seconds.
WARN: [App Key Value Store migration] Service(36) terminated before the service availability check could complete. Exit code 14, waited for 2 seconds.
App Key Value Store migration failed, check the migration log for details. After you have addressed the cause of the service failure, run the migration again, otherwise App Key Value Store won’t function.
[App Key Value Store migration] Checking if migration is needed. Upgrade type 2. This can take up to 600seconds.
WARN: [App Key Value Store migration] Service(36) terminated before the service availability check could complete. Exit code 14, waited for 0 seconds.
App Key Value Store migration failed, check the migration log for details. After you have addressed the cause of the service failure, run the migration again, otherwise App Key Value Store won’t function.
[DFS] Performing migration.
[DFS] Finished migration.

Can somebody help me please? 

I already tried to delete the mongod.lock file and run the upgrade again but it didn't work 

[s2_splunk]

Can you follow the steps in the accepted solution here and see if it addresses the KVStore upgrade issue? If not, mongod.log would be helpful to see.

[daymar_23]

Hello @s2_splunk 

Here are the steps to workaround the issue:
1. Stop Splunk.
2. Set server.conf:listenOnIPv6 = no.
3. Perform KVstore migration by executing: ./splunk migrate migrate-kvstore.
4. Set server.conf:listenOnIPv6 = yes.
5. Start Splunk.

Do I have to start splunk and after that try again with the Update from 7.2.1 to 8.1.4 not working? Or before start splunk? 

[s2_splunk]

If you completely backed out your prior upgrade, I think you could try and replace step #3 with the upgrade itself. But honestly, I am not sure. I am not part of our support team. If you have a valid support agreement and can't get this to work, please open a support case as well.

[daymar_23]

Well, it doesn't work 😞 

I will submit a case with the support team to know what else I can do. 

Thanks for your help @s2_splunk