Why is Splunk UF version 9.0.1 stuck while accepting license?

[nbommu]

We have existing UF 8.2.2 in all instance and managed by ansible, when we are trying upgrade to 9.0.1 ansible stuck which accepting license, if I run that command manually on the target host, it is working fine, I think ansible is waiting for response after ran the command on the target hots which has changed with version 9.0.1.

task to accept license:

- name: Accept Splunk license and set up init script
  command:
    cmd: /opt/splunkforwarder/bin/splunk status --accept-license --answer-yes --no-prompt

[nbommu]

with version 9.x.x, here is the command option to accept the license and it work via ansible.

`/opt/splunkforwarder/bin/splunk start --accept-license --answer-yes`

[Jordan1]

Hi all,

Has anyone found a solution to this issue as I've had a similar experience. I'm using Tanium to upgrade Splunk UF's and getting this same issue of the license not getting accepted on Linux.

I also opened a Splunk support case which got closed due to Splunk saying the issue was with Tanium.

I have tried adding the KVStore stanza to the server.conf and disabling = true but this hasn't made any difference.

[VatsalJagani]

@nbommu , @schose - Note that there was a issue like that with Splunk 9.0.0 and 9.0.1 which was fixed in 9.0.2 or 9.0.3.

Release notes - https://docs.splunk.com/Documentation/Splunk/9.0.1/ReleaseNotes/KnownIssues

Please consider upvoting/accepting the answer if this helps!!!

[likedasplunk]

Yeah,

9.0.3 still has this exact same issue. This issue is definitely NOT fixed.

[nbommu]

I opened a support ticket with all the details, they are redirecting to ansible since it was stuck there, even I clearly mentioned that it is not ansible issue and there is something to do with version 9, support is eager to close the ticket instead of understanding the issue and debug 🙂

[likedasplunk]

@nbommu  This isn't an issue with Ansible either. I use the installer in a bash script and it does the same thing. I try to just install and use the switches to accept the license manually and it does the same thing.

It's a bug in the nix UF installer. 

[VatsalJagani]

@likedasplunk - Have you tried disabling the kvstore from the server.conf on the UF as suggested by @schose ?

[likedasplunk]

@VatsalJagani 

I did try disabling kvstore in server.conf and it did not fix the issue.

[Jordan1]

Did you find a solution to this?

As I'm experiencing the same issue

[likedasplunk]

@Jordan1 

I did not.

Splunk support is actually telling me that they don't have any issues in their "lab". I've sent over all the documentation and screenshots that I can take. I have a meeting with them some time tomorrow, but there's not doubt that this is still an issue.

The suggested work arounds here do not work. I'm almost certain that this is a bug in the UF installer.

[Jordan1]

Hi @likedasplunk 

I had a call a few weeks ago as well with Splunk Support. Please let me know if you find the solution from them. If I find anything I will let you know

[Jordan1]

Hi @likedasplunk 

Did you find any solution with Splunk Support?

[likedasplunk]

@Jordan1 @schose  @nbommu @VatsalJagani 

I was out for the holiday yesterday. I'm trying to set up a new time with support. I'm fairly certain that this is a bug in the Nix UF. During an upgrade last week, we had the exact same issue with Splunk Enterprise. The only way to bypass the whole license page was to do this for the Splunk Enterprise upgrade install:

echo y | ./splunk start --accept-license

[likedasplunk]

So, I think the --no-prompt switch was causing this issue. Assuming you're running Splunk as the user 'splunk', using this line in my install/upgrade script now works:

runuser -l splunk -c '/opt/splunkforwarder/bin/splunk start --accept-license --answer-yes'

[nbommu]

any luck with splunk support?

[schose]

Hi,

Same here.. for some reason the update script tries to upgrade mongo-db on UFW and get stuck. seems like a bug.

disable kvstore in server.conf works for me.

https://community.splunk.com/t5/Splunk-Enterprise/Splunk-9-Universal-Forwarder-getting-quot-App-Key-...

regards,

Andreas

[likedasplunk]

Hi @schose 

I think @nbommu's original post is about getting stuck at the license screen while using the switches to accept the license during an automated install, not for the kvstore issue. We might be talking about two different things here.

[nbommu]

thanks for checking Andreas, looks like I'm not alone 🙂

I opened kvstore bug as well as ansible, let see if we get quick workaround on them.

Thanks

Niranjan

[schose]

Hi Niranjan,

Mongo is no component started by UFW but only by Splunk Enterprise.. So this is surely a bug.. it also see to make a difference if you use .tgz or .deb. 

I had the issue using .tgz. My workaround is this task:

- name: ensure kvstore is disabled on UFW
  ini_file: 
    dest: /opt/splunkforwarder/etc/system/local/server.conf 
    section: kvstore 
    option: disabled 
    value: true

regards,

Andreas

[nbommu]

thanks Andreas.

I was able to start Splunk UF ver 9.0.1 after disabling kvstore and accepting license manually, but stuck while accepting license via anisble, will post if I have more info from splunk support on this.

thanks

Niranjan