appIcon.png not displayed in Splunk GUI for custom and default apps

[dkolekar_splunk]

In Splunk search head, the app logos are not visible. And If we click on app > right click "copy link address", the customer is getting 500 internal server error. This issue is happening with all users.

1. The appIcon.png permissions are correct
2. We deleted-restarted/reinstated-restarted appIcon.png but no change.
3. Checked the validate command - $Splunk_Home/bin ./splunk validate files Output: Validating installed files against hashes from '/opt/splunk/splunk-6.3.12-911c7597cdd8-linux-2.6-x86_64-manifest' All installed files intact.
4. While investigating, I found that under the app directory the static directory is missing. Path: $Splunk_Home/etc/app//static. So, I copied $Splunk_Home/etc/app//appserver/static to $Splunk_Home/etc/app//static. But no change.

[dkolekar_splunk · Accepted solution]

To investigate this issue try following.
1. Permission of appIcon.png file should be correct
2. Path of the appIcon.png file should be correct
3. Check if you are getting any error in web_acess.log & splunkd_access.log
In my case, I found,
web_acess.log

10.185.8.19 - 507394
[08/Apr/2019:11:08:28.688 +0100]
"GET /en-GB/splunkd/__raw/servicesNS/507394/abc/static/appIcon.png HTTP/1.1" 500 2996
"" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/14215454 Safari/16515.36" - 5ckjjbfhrbgergvu9c1aded0 11ms

splunkd_access.log

127.0.0.1 - 507394
[08/Apr/2019:11:08:28.695 +0100]
"GET /servicesNS/507394/abc/static/appIcon.png HTTP/1.0" 200 20386 - - - 1ms

appIcon.png request has valid response code 200 in splunkd_access.log and in web_access.log it is throwing 500 status error code.
Under $Splunk_Home/etc/system/local/web.conf I found following setting.

[settings]
appServerPorts = 0

appServerPorts = positive integer
Port number(s) for the Python-based application server to listen on.
This port is bound only on the loopback interface -- it is not
exposed to the network at large.
If set to "0", prevents the application server from
being run from splunkd. Instead, Splunk Web starts as
a separate python-based service which directly listens to the
'httpport'. This is how Splunk 6.1.X and earlier behaved.

After removing this, the issue got resolved.