Deployment Architecture

After upgrading a search head cluster to Splunk 6.3, why are all our launcher app icons missing?

Motivator

After upgrading to 6.3 (search head clustering) all our launcher app icons have disappeared even for default untouched apps.

ie. search & reporting.

alt text

The path to the icon shows an "undefined" path.

 <img src="/en-US/splunkd/__raw/servicesNS/undefined/search/static/appIcon.png">

Upgrade was performed by stopping all instances.

Upgrading deployer.
Restart deployer.
Upgrade all search head cluster member instances.
Start member instances.
Wait for 5 minutes.
Do a new deploy from deployer.

1 Solution

Splunk Employee
Splunk Employee

As previously posted this is a known issue SPL-107449 where the UI is missing app icons and navigation drop downs.

It has been reported in 6.3 and 6.3.1

http://docs.splunk.com/Documentation/Splunk/6.3.1/ReleaseNotes/Knownissues

This is related from python 2.7.8 upgrade to python 2.7.9 where setting a cookie with brackets in the name can cause the issue.
Python has reported this bug in 2.7.9 and fixed in 2.7.10 as referenced in this link:
https://bugs.python.org/issue22931

Below are 2 options that can be used prior to the fix which is tentatively targeted for the next 6.3.x maintenance release.
Note: When the maintenance release is applied, it will overwrite this modified Cookie.py

Option 1:

1) Make a backup copy of $SPLUNK_HOME/lib/python2.7/Cookie.py

Keep in mind if for some reason you have custom changes in this Cookie.py, it will be overwritten by the replacement file for this fix in an upcoming maintenance release

2) Modify $SPLUNK_HOME/lib/python2.7/Cookie.py as the user who starts Splunk with the following changes:

From:

_LegalCharsPatt = r"[\w\d!#%&'~_`><@,:/\$\*\+\-\.\^\|\)\(\?\}\{\=]"

To:

_LegalCharsPatt = r"[\w\d!#%&'~_`><@,:/\$\*\+\-\.\^\|\)\(\?\[\]\}\{\=]"

3) Restart your Splunk search head(s)

Option 2:

1) Download the modified Cookie.py from here that includes the same fix as in Option 1
If you have issues with the download, please contact Support.

2) Make a backup copy of $SPLUNK_HOME/lib/python2.7/Cookie.py

3) Replace $SPLUNK_HOME/lib/python2.7/Cookie.py with the modified file

4) Restart your Splunk search head(s)

View solution in original post

Splunk Employee
Splunk Employee

There was no change in the location of Cookie.py under 6.3.1
It remains to be found in $SPLUNK_HOME/lib/python2.7

0 Karma

Splunk Employee
Splunk Employee

For completeness:
6.3 and 6.3.1

Linux/Solaris/Mac OS

$SPLUNK_HOME/lib/python2.7/Cookie.py

Windows

%SPLUNK_HOME%\Python-2.7\lib\Cookie.py
0 Karma

Path Finder

As I said, I just installed 6.3.1 (on Windows). $SPLUNK_HOME\lib has no Python 2.7 subfolder. The $SPLUNK_HOME\Python 2.7 subfolder does have a lib subfolder and Cookie.py is located there. I am getting this info by looking in the file system of a fresh install (10 fresh Windows installs to be exact). All 10 were installed from the archive located at the splunk download page (http://www.splunk.com/en_us/download/splunk-enterprise.html). I made no changes in the directory structure. Updating the Cookie.py file at $SPLUNK_HOME\Python 2.7\lib had the desired effect.

0 Karma

Splunk Employee
Splunk Employee

This differs between linux and windows installs.

splunk-6.3.0-aa7d4b1ccb80-Linux-x86_64> find . -name Cookie.py
./lib/python2.7/Cookie.py

PS C:\Users\winders\Desktop\splunk-6.3.0-aa7d4b1ccb80-windows-64> Get-ChildItem -Path . -Name Cookie.py -Recurse
splunk\Python-2.7\Lib\Cookie.py
0 Karma

Path Finder

That leads to the question "why?" That question should be a separate topic though. I just wanted to be sure that this question had a more complete answer since Support is referring people to it.

0 Karma

Path Finder

This worked for me. Thanks!

Explorer

I Modify $SPLUNK_HOME/lib/python2.7/Cookie.py as below ,but it doesn't work,icons are still missing

Splunk Employee
Splunk Employee

Please log case with Splunk Support and they will have suggestion for you.

0 Karma

Explorer

6.3.1 doesn't seem to have fixed this bug either. Anyone found a permanent workaround for this?

0 Karma

Explorer

I'm commenting here hoping that my experience might help someone else who is experiencing my variation of this problem, which initially manifested in the same way OP illustrated (which brought me to this post), but was partially fixed by force reloading the broken pages. The part that wasn't fixed with the cookie clearing method involved broken menus as a persistent issue.

After I upgraded, I initially had the same broken images along with broken menus, using Firefox or Chrome for the instances which I did NOT have an Apache proxy handling auths, but the images portion of the problem didn't last long. I believe it was fixed because I have a habit of force reloading a webpage (Ctrl+F5) anytime I see broken images or menus.

With that said, even though the broken images were fixed, there continued to be certain pages/locations where the navbars and appbars are just broken, and to confuse me even more, it magically started working once, for like 2 minutes, only to break again shortly afterwards. The fastest repeatable test I found was navigating to the indexer clustering page on an instance which didn't have clustering configured yet. I tried clearing cookies, not using the fqdn, using the IP instead of the dns name, navigating to/from different pages with different contexts, modifying Cookie.py as instructed by rbal and a few other random things. Eventually I created a new host with a fresh install, thinking that running browsers side-by-side with a working instance would give me some clues to how I could fix it, but I was bummed to find that the fresh install was exhibiting the same behavior.

When thinking about my working instances, I thought that Apache handling the auth was somehow the fix, so I was going to install and configure it on my test instance, but before I did that, I realized that I had overlooked one Splunk setting which was different regardless of the proxy. The non-proxied systems weren't using https. Finally! Enabling Splunk Web SSL, and logging in with the https prefix fixed the menus... disabling it and logging in with the http prefix broke them again. Toggled back and forth between SSL and not a few times with consistent results.

Anyways, I know it was a bit long winded, but I hope it helps someone.

0 Karma

Splunk Employee
Splunk Employee

This symptom is being tracked by Splunk Bug

SPL-107449: After upgrade to 6.3 - Customer is missing some tabs from Web GUI

0 Karma

Splunk Employee
Splunk Employee

As a workaround you can try this ,,,,,

removed by Koops

0 Karma

Motivator

rbal, this answer was marked as introducing a security risk in a previous thread.

I removed it due to security reasons.

Engager

Our installation has also had the same issue, specific to Chrome.

I saw "undefined" all over the place, where there should have been my username instead, including in references to Javascript. Deleting cookies was the permanent solution. Incognito mode or using Safari was a temporary "fix".

0 Karma

New Member

we're still having issues after clearing cookies, so not a permanent solutions for us.

0 Karma

New Member

Having the same issues after upgrading.

This looks like an issues with cookies and Chrome only. And is connected to other cookies on main domain. Not sure how to solve it yet.

0 Karma

New Member

I was having the same problem. I deleted all my browser cookies and it started working.

However, other users on my team are still having the problem, even after deleting their cookies.

0 Karma

Path Finder

I too have the same issues after upgrading from 6.2.5 (w/SHC enabled) to 6.3, and was hoping to spur some thoughts by sharing how I am able to reproduce it...

It doesn't matter which browser I choose, but adding the domain name to my host in the URL seems to cause the issues noted (missing icons, menus). When I remove domain, it magically works.

https://splunk (works perfectly)
https://splunk. (missing icons, menus)

0 Karma

Builder

I just tried connecting by IP address, and that seems to work for me.

0 Karma

Motivator

alt text

It is broken with the latest version of chrome also.

0 Karma

Contributor

Hello, Lucas K
we have similar case with control buttons and pickers.
have you issue only with app icons?

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!