500s when updating apps from GUI with walkaround

[PickleRick]

Hello there.

Posting just for reference.

It seems there is some misconfguration issue between splunkbase and the Splunk default config.

The default config says:

# /opt/splunk/bin/splunk btool server list applicationsManagement | grep updateHost
updateHost = https://apps.splunk.com

# /opt/splunk/bin/splunk btool server list applicationsManagement | grep Check
sslAltNameToCheck = splunkbase.splunk.com, apps.splunk.com, cdn.apps.splunk.com
sslCommonNameToCheck = apps.splunk.com, cdn.apps.splunk.com

However, the servers respond with:

# curl -v https://apps.splunk.com 2>&1 | grep subject:
* subject: C=US; ST=California; L=San Francisco; O=Splunk Inc.; CN=splunkbase.splunk.com

Whereas 8.2.5 (don't have any other 8.2 at hand to check) seems to work despite those settings, 9.0.3 enforces the settings strictly and says

ERROR X509 [25665 TcpChannelThread] - X509 certificate (CN=splunkbase.splunk.com,O=Splunk Inc.,L=San Francisco,ST=California,C=US) common name (splunkbase.splunk.com) did not match any allowed names (apps.splunk.com,cdn.apps.splunk.com)

Walkaround:

Overwrite the setting in server.conf with

[applicationsManagement]
sslCommonNameToCheck = splunkbase.splunk.com,apps.splunk.com,cdn.apps.splunk.com

[nzou]

I tried to override the settings in server.conf and restart the Splunk Enterprise, seems I still get 'uncaught exception"

And saw this in browser console:

common.js:1349 
        
        
       POST http://localhost:8000/en-US/splunkd/__raw/services/apps/local 500 (Internal Server Error)

Tried to override some other settings under `applicationsManagement`, seems won't work...

And the error in _internal log sounds not useful at all.

12-07-2023 13:54:14.770 -0800 ERROR ApplicationUpdater [2903300 TcpChannelThread] - Unexpected error downloading update: Uncaught exception

[PickleRick]

Check your effective config with btool to see if you've successfully overriden the settings.

But you may also be hitting some different issue.

[nzou]

Checked, but seems there are some other issues. But the 'Uncaught Exception' error really doesn't help... 😞

Checked the splunkd log and python log as well, nothing special....

Contacted splunk support, until now, no update on this issue yet.

[PickleRick]

Might be that there is another issue indeed. Keep us posted if there is something potentially hiting other users as well going on.

[nzou]

This issue should have been solved by Splunk. Now, I can download and install my app without any problems. FYI.

[jhupka_splunk]

I'm responding to add a +1 to this issue from a new Splunkbase app install perspective versus updating an existing app already installed.  Did your issue prevent you from updating the app?  I was able to install new Splunkbase apps fine, but still got the same X509 errors you 

Details:

I started with a fresh Splunk 9.0.3 install on Mac OSX, and I ran the same btool commands and was configured like you.

I tried installing a Splunkbase app from the Splunk GUI, and I found this in index=_internal after installing:

02-13-2023 11:44:52.351 -0700 ERROR X509 [2149013 TcpChannelThread] - X509 certificate (CN=splunkbase.splunk.com,O=Splunk Inc.,L=San Francisco,ST=California,C=US) common name (splunkbase.splunk.com) did not match any allowed names (apps.splunk.com,cdn.apps.splunk.com)
I was able to install a fresh copy of Splunk 9.0.3 and install an app from Splunkbase that I needed without any issues.

The app still installed with no issues...but I do see that chatty error.

I added a stanza like you to etc/system/local/server.conf and restarted Splunk:

[applicationsManagement]
sslCommonNameToCheck = splunkbase.splunk.com,apps.splunk.com,cdn.apps.splunk.com

 This time, when I installed a Splunkbase app from the Splunk GUI I didn't see any more of those X509 ERROR logs.

[PickleRick]

I could _not_ update the app. The update package would not download I assume. The UI would tell me that update failed with error 500. I could, however, see the update as available.