All Apps and Add-ons

Issues with Splunk Workday Add-on, not saving configuration, spinning wheel 'Loading'

glowry
New Member

We had the workday app (4.1) running on a HF (6.6.5) that apparently stopped working on Oct 24th.

Error:
File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\packages\splunklib\binding.py", line 1221, in request
raise HTTPError(response)
HTTPError: HTTP 503 Service Unavailable -- KV Store initialization failed. Please contact your system administrator.

I am now attempting to move that add-on to a new HF...as recommended by support for another case.

  1. I have copied the TA from the HF that was working, restarted Splunk - it didnt work.
  2. I have re-installed v 4.1 on the HF from the .tgz file, restarted - it didnt work
  3. I have re-installed with v 5.0 on the HF from .tgz, restarted - it didnt work
  4. I've pushed the security settings down from the \etc\apps folder down to all subfolders...to eliminate any permissions issues.

When i log into the web ui, go to workday app, then go to configuration> add-on settings...the loading icon appears and shows about 4-5 bars and is stuck...it eventually shows the setting options, i configure them, hit save, but it never saves them.

The ta never attempts to call out to the internet to grab files, looking in the workday ta log, the splunkd and splunkaccess logs i see 404 and 500 http server errors.

splunkd.log:
11-07-2018 01:18:53.847 -0600 ERROR AdminManagerExternal - Unexpected error "" from python handler: "REST Error [500]: Internal Server Error -- Traceback (most recent call last):\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\handler.py", line 113, in wrapper\n for name, data, acl in meth(self, *args, **kwargs):\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\handler.py", line 299, in _format_response\n masked = self.rest_credentials.decrypt_for_get(name, data)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\credentials.py", line 184, in decrypt_for_get\n clear_password = self._get(name)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\credentials.py", line 389, in _get\n string = mgr.get_password(user=context.username())\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\utils.py", line 154, in wrapper\n return func(*args, **kwargs)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\credentials.py", line 118, in get_password\n all_passwords = self._get_all_passwords()\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\utils.py", line 154, in wrapper\n return func(*args, **kwargs)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\credentials.py", line 272, in _get_all_passwords\n clear_password += field_clear[index]\nTypeError: cannot concatenate 'str' and 'NoneType' objects\n". See splunkd.log for more details.

11-07-2018 01:19:05.427 -0600 ERROR AdminManagerExternal - Unexpected error "" from python handler: "REST Error [500]: Internal Server Error -- Traceback (most recent call last):\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\handler.py", line 113, in wrapper\n for name, data, acl in meth(self, *args, **kwargs):\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\handler.py", line 299, in _format_response\n masked = self.rest_credentials.decrypt_for_get(name, data)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\credentials.py", line 184, in decrypt_for_get\n clear_password = self._get(name)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\credentials.py", line 389, in _get\n string = mgr.get_password(user=context.username())\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\utils.py", line 154, in wrapper\n return func(*args, **kwargs)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\credentials.py", line 118, in get_password\n all_passwords = self._get_all_passwords()\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\utils.py", line 154, in wrapper\n return func(*args, **kwargs)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\credentials.py", line 272, in _get_all_passwords\n clear_password += field_clear[index]\nTypeError: cannot concatenate 'str' and 'NoneType' objects\n". See splunkd.log for more details.

11-07-2018 01:23:52.873 -0600 ERROR AdminManagerExternal - Unexpected error "" from python handler: "REST Error [500]: Internal Server Error -- Traceback (most recent call last):\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\handler.py", line 113, in wrapper\n for name, data, acl in meth(self, *args, **kwargs):\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\handler.py", line 299, in _format_response\n masked = self.rest_credentials.decrypt_for_get(name, data)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\credentials.py", line 184, in decrypt_for_get\n clear_password = self._get(name)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\credentials.py", line 389, in _get\n string = mgr.get_password(user=context.username())\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\utils.py", line 154, in wrapper\n return func(*args, **kwargs)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\credentials.py", line 118, in get_password\n all_passwords = self._get_all_passwords()\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\utils.py", line 154, in wrapper\n return func(*args, **kwargs)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\credentials.py", line 272, in _get_all_passwords\n clear_password += field_clear[index]\nTypeError: cannot concatenate 'str' and 'NoneType' objects\n". See splunkd.log for more details.

11-07-2018 01:24:02.461 -0600 ERROR AdminManagerExternal - Unexpected error "" from python handler: "REST Error [500]: Internal Server Error -- Traceback (most recent call last):\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\handler.py", line 113, in wrapper\n for name, data, acl in meth(self, *args, **kwargs):\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\handler.py", line 299, in _format_response\n masked = self.rest_credentials.decrypt_for_get(name, data)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\credentials.py", line 184, in decrypt_for_get\n clear_password = self._get(name)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\credentials.py", line 389, in _get\n string = mgr.get_password(user=context.username())\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\utils.py", line 154, in wrapper\n return func(*args, **kwargs)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\credentials.py", line 118, in get_password\n all_passwords = self._get_all_passwords()\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\utils.py", line 154, in wrapper\n return func(*args, **kwargs)\n File "D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\credentials.py", line 272, in _get_all_passwords\n clear_password += field_clear[index]\nTypeError: cannot concatenate 'str' and 'NoneType' objects\n". See splunkd.log for more details.

11-07-2018 01:24:02.651 -0600 ERROR ExecProcessor - message from "python "D:\Program Files\Splunk\etc\apps\TA-workday\bin\workday.py"" HTTPError: HTTP 500 Internal Server Error -- {"messages":[{"type":"ERROR","text":"Unexpected error \"\" from python handler: \"REST Error [500]: Internal Server Error -- Traceback (most recent call last):\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\handler.py\", line 113, in wrapper\n for name, data, acl in meth(self, *args, **kwargs):\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\handler.py\", line 299, in _format_response\n masked = self.rest_credentials.decrypt_for_get(name, data)\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\credentials.py\", line 184, in decrypt_for_get\n clear_password = self._get(name)\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\credentials.py\", line 389, in _get\n string = mgr.get_password(user=context.username())\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\utils.py\", line 154, in wrapper\n return func(*args, **kwargs)\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\credentials.py\", line 118, in get_password\n all_passwords = self._get_all_passwords()\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\utils.py\", line 154, in wrapper\n return func(*args, **kwargs)\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\credentials.py\", line 272, in _get_all_passwords\n clear_password += field_clear[index]\nTypeError: cannot concatenate 'str' and 'NoneType' objects\n\". See splunkd.log for more details."}]}

11-07-2018 01:24:03.173 -0600 ERROR ExecProcessor - message from "python "D:\Program Files\Splunk\etc\apps\TA-workday\bin\workday.py"" ERRORHTTP 500 Internal Server Error -- {"messages":[{"type":"ERROR","text":"Unexpected error \"\" from python handler: \"REST Error [500]: Internal Server Error -- Traceback (most recent call last):\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\handler.py\", line 113, in wrapper\n for name, data, acl in meth(self, *args, **kwargs):\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\handler.py\", line 299, in _format_response\n masked = self.rest_credentials.decrypt_for_get(name, data)\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\credentials.py\", line 184, in decrypt_for_get\n clear_password = self._get(name)\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\splunktaucclib\rest_handler\credentials.py\", line 389, in _get\n string = mgr.get_password(user=context.username())\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\utils.py\", line 154, in wrapper\n return func(*args, **kwargs)\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\credentials.py\", line 118, in get_password\n all_passwords = self._get_all_passwords()\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\utils.py\", line 154, in wrapper\n return func(*args, **kwargs)\n File \"D:\Program Files\Splunk\etc\apps\TA-workday\bin\ta_workday\solnlib\credentials.py\", line 272, in _get_all_passwords\n clear_password += field_clear[index]\nTypeError: cannot concatenate 'str' and 'NoneType' objects\n\". See splunkd.log for more details."}]}

splunkaccess.log
127.0.0.1 - splunk-system-user [06/Nov/2018:15:48:45.336 -0600] "GET /servicesNS/nobody/TA-workday/properties/TA-workday HTTP/1.1" 404 143 - - - 1ms

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Splunk Education - Fast Start Program!

Welcome to Splunk Education! Splunk training programs are designed to enable you to get started quickly and ...

Five Subtly Different Ways of Adding Manual Instrumentation in Java

You can find the code of this example on GitHub here. Please feel free to star the repository to keep in ...

New Splunk APM Enhancements Help Troubleshoot Your MySQL and NoSQL Databases Faster

Splunk Observability has two new enhancements to make it quicker and easier to troubleshoot slow or frequently ...