Just upgraded from 6.1.1 to 6.2.2 (splunk-6.2.2-255606-linux-2.6-x86_64.rpm) on a RHEL 5.7.
At the login page, any valid login/password pair returns a "Server error" red message and splunkd.log reports:
03-09-2015 09:50:00.696 +0100 ERROR UiAuth - user=admin action=login status=failure reason=bad-cval useragent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:36.0) Gecko/20100101 Firefox/36.0" clientip=10.1.2.3
We have not found previous references to the "bad-cval" error. Typing a wrong password returns an auth failure, therefore it is something happening after the login validation.
What should we look for?
Following my last comment, I retouched the configuration.
In $SPLUNK_HOME/etc/system/local/web.conf I had defined this:
root_endpoint = /splunk/
As soon as I edited it into
root_endpoint = /splunk
and restarted ... user access was restored.
Curiously this worked up to 6.1 and broke on 6.2.2, probably with the removal of SplunkWeb process (?).
had the same issue with splunk 6.5.3
in my case the problem was the firefox extension priv3+
Following my last comment, I retouched the configuration.
In $SPLUNK_HOME/etc/system/local/web.conf I had defined this:
root_endpoint = /splunk/
As soon as I edited it into
root_endpoint = /splunk
and restarted ... user access was restored.
Curiously this worked up to 6.1 and broke on 6.2.2, probably with the removal of SplunkWeb process (?).
Going further with debug, I activated ALL possible debugs and filtering out events I came up with this:
03-09-2015 15:35:10.609 DEBUG HTTPAuthManager - login...
03-09-2015 15:35:10.609 DEBUG AuthenticationManagerSplunk - Attempting login for user 'admin'
03-09-2015 15:35:10.613 DEBUG UserManagerPro - user="admin" successfully logged in
03-09-2015 15:35:10.613 DEBUG HTTPAuthManager - login successful
03-09-2015 15:35:10.613 INFO UserManager - Setting user context: splunk-system-user
03-09-2015 15:35:10.613 INFO UserManager - Done setting user context: NULL -> splunk-system-user
03-09-2015 15:35:10.613 DEBUG AuthenticationManagerSplunk - Getting info for user: admin
03-09-2015 15:35:10.613 DEBUG AuthenticationManagerSplunk - Getting info for user: admin
03-09-2015 15:35:10.613 DEBUG AuthenticationManagerSplunk - Getting info for user: admin
03-09-2015 15:35:10.613 INFO UserManager - Unwound user context: splunk-system-user -> NULL
03-09-2015 15:35:10.613 ERROR UiAuth - user=admin action=login status=failure reason=bad-cval useragent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:36.0) Gecko/20100101 Firefox/36.0" clientip=10.1.2.3
I doubt I can get more info out of debug logs. If I am wrong, please suggest how to proceed. This is a live, production, Splunk instance with GBs of logs per day, non-stop indexing.
stand alone "all in one box" or what is your environment like?
LDAP configured or splunk auth?
did you try other browsers?
other users having same issue?
Thank you for jumping in.
It's a all-in-one-box that worked until we upgraded to 6.2.2. We use LDAP auth, but "admin" has always authenticated against the Splunk internal password db. Neither admin nor LDAP accounts can now access Splunk webUI. Tried IE (first time it met Splunk) and FF36 on Win7, with clean cookies: "Server error".
The server is not swapping to disk. I am currently at home and cannot provide other details. Please let me know which other environment info could be useful.
What is the locale in the URL?
Like mine is "en-US" here "http://searchhead:8000/en-US/app/launcher/home"
When I call "http://searchhead:8000/en-US/app/launcher/home" I get redirected to
"http://searchhead:8000/splunk**//**en-US/account/login?return_to=%2Fsplunk%2Fen-US%2Fapp%2Flauncher%...". Note the double slash before the locale.
If I manually edit the URL and let only one slash then access is possible.
Problem is: when I log out I am thrown again at an URL with double slash, that won't let me in.
We can correct our bookmarks, but it worked with all Splunk versions throughout 6.1.1 and broke on 6.2.2. Is there a config setting to tweak?
This happens with all locales I tried: en-US, en-GB, it-IT.
"cval" is one of the cookies set by Splunk. But even cleaning up the browser doesn't solve the issue. The cookie is regenerated (sample content 657767914).