License keys disappear after restart

[Snorre]

I added a 30gb renewal license key that is valid from June 14th later this year. Afterward I got a message telling me to restart Splunk. I did that, and now all other licenses are missing from the License admin console. Anyone experienced this before? Is there a way to recover the old licenses?

Running splunk Enterprise 9.2.0.1 on prem on redhat

[PickleRick]

Normally the licenses shouldn't "disappear" on their own. Even when licenses expire, they still show as expired. The licenses are backed by files in $SPLUNK_HOME/etc/licenses so if they "disappeared" someone must have deleted them. Check your backups for contents of this directory.

[Snorre]

Thanks for getting back to me.

I started to look into the /etc/licenses folder and I toyed around with the files there, and now I think I figured out what is happening: If I install a Dev key in the Prod environment, splunk deletes all Prod keys in the folder and creates a Restart required message in the dashboard. After restart only the files installed after the dev key is loaded. 

I might very well have installed a new dev key in the prod environment as I received a renewal keys for both prod and dev in the same email.

We will ask the maintenance team for a restore of the files in the licenses folder and it will probably be sorted.

It would be great if splunk could show a warning when I try to do such stupid things as uploading a dev license in the prod environment, or maybe even a backup of the license files when deleting them, but I have learned now and wont be doing that again 🙂

[PickleRick]

Ahh... right. If you change the license type, that might indeed cause "strange" behaviour since different license types normally don't stack and may enable different features. Hence the restart.

[livehybrid]

Hi @Snorre 

The license files are XML files inside, so if you have a look at the contents of the file in the license directory you might be able to clarify which one you applied if unsure. They each have a unique signature (amongst other things) inside the file. Any text edit should work for viewing them. 

🌟 Did this answer help you? If so, please consider:

• Adding kudos to show it was useful

• Marking it as the solution if it resolved your issue

• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing.