500 Internal Error while user sso to Splunk

[ben_leung]

Trying to access Splunk search head. One of the search heads gives this error when trying to access via sso.

Splunk UI

500 Internal Server Error

Return to Splunk home page

An error occurred while rendering the page template. See web_service.log for more details

web_service.log

index=_internal sourcetype=splunk_web_service host=searchhead2

3 events when trying to access the host

2014-06-09 19:39:54,498 INFO    [53960d8a7d3778d50] cached:77 - memoized decorator used on function <function getEntities at 0x2126f50> with non hash able arguments

2014-06-09 19:39:54,703 ERROR   [53960d8a7d3778d50] config:81 - [HTTP 403] Client is not authorized to perform requested action; https://127.0.0.1:8089/services/search/timeparser/tz
Traceback (most recent call last):
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/config.py", line 79, in getServerZoneInfo
    return times.getServerZoneinfo()
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/times.py", line 158, in getServerZoneinfo
    serverStatus, serverResp = splunk.rest.simpleRequest('/search/timeparser/tz')
  File "/opt/splunk/lib/python2.7/site-packages/splunk/rest/__init__.py", line 483, in simpleRequest
    raise splunk.AuthorizationFailed(extendedMessages=uri)
AuthorizationFailed: [HTTP 403] Client is not authorized to perform requested action; https://127.0.0.1:8089/services/search/timeparser/tz

2014-06-09 19:39:54,718 ERROR   [53960d8a7d3778d50] __init__:281 - Mako failed to render: 
Traceback (most recent call last):
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/controllers/__init__.py", line 277, in render_template
    return templateInstance.render(**template_args)
  File "/opt/splunk/lib/python2.7/site-packages/mako/template.py", line 283, in render
    return runtime._render(self, self.callable_, args, data)
  File "/opt/splunk/lib/python2.7/site-packages/mako/runtime.py", line 575, in _render
    **_kwargs_for_callable(callable_, data))
  File "/opt/splunk/lib/python2.7/site-packages/mako/runtime.py", line 607, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/opt/splunk/lib/python2.7/site-packages/mako/runtime.py", line 633, in _exec_template
    callable_(context, *args, **kwargs)
  File "/opt/splunk/share/splunk/search_mrsparkle/templates/layout/base.html", line 10, in render_body
    <%self:render/>
  File "/opt/splunk/share/splunk/search_mrsparkle/templates/layout/base.html", line 25, in render_render
    ##
  File "/opt/splunk/share/splunk/search_mrsparkle/templates/layout/base.html", line 67, in render_pagedoc
    % else:
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/config.py", line 103, in getConfig
    args.update(_get_app_config(namespace))
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/config.py", line 48, in _get_app_config
    rawConfig = splunk.bundle.getConf('app', namespace=app)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/bundle.py", line 39, in getConf
    serverResponse, serverContent = rest.simpleRequest(uri, getargs={'fillcontents':1}, sessionKey=sessionKey)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/rest/__init__.py", line 483, in simpleRequest
    raise splunk.AuthorizationFailed(extendedMessages=uri)
AuthorizationFailed: [HTTP 403] Client is not authorized to perform requested action; https://127.0.0.1:8089/servicesNS/509693468/search/properties/app?fillcontents=1

[ben_leung · Accepted solution]

Search heads were misconfigured. Capabilities for the admin role were not inheriting user, which had the rest_properties_* to access the endpoint. Had a fellow colleague set the admin role straight.