App Issues on Splunk Heavy Forwarder

[ZacEsa]

I'm trying to get my app input coming in via a heavy forwarder. I've deployed the app to the heavy forwarder and configured the necessary but, I'm seeing these logs in my splunkd.log file in the heavy forwarder.

07-19-2017 18:06:36.599 +0800 ERROR PasswordHandler - Decrypted password from stanza=credential:__REST_CREDENTIAL__#TA-Cb_Defense#configs/conf-ta_cb_defense_settings:additional_parameters``splunk_cred_sep``1: is not utf8, skipping
07-19-2017 18:06:36.600 +0800 ERROR PasswordHandler - Decrypted password from stanza=credential:__REST_CREDENTIAL__#TA-Cb_Defense#configs/conf-ta_cb_defense_settings:additional_parameters``splunk_cred_sep``2: is not utf8, skipping
07-19-2017 18:06:36.642 +0800 ERROR AdminManagerExternal - Stack trace from python handler:\nTraceback (most recent call last):\n  File "/opt/splunk/lib/python2.7/site-packages/splunk/admin.py", line 130, in init\n    hand.execute(info)\n  File "/opt/splunk/lib/python2.7/site-packages/splunk/admin.py", line 594, in execute\n    if self.requestedAction == ACTION_LIST:     self.handleList(confInfo)\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/splunk_aoblib/rest_migration.py", line 38, in handleList\n    AdminExternalHandler.handleList(self, confInfo)\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/splunktaucclib/rest_handler/admin_external.py", line 40, in wrapper\n    for entity in result:\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/splunktaucclib/rest_handler/handler.py", line 120, in wrapper\n    raise RestError(500, traceback.format_exc())\nRestError: REST Error [500]: Internal Server Error -- Traceback (most recent call last):\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/splunktaucclib/rest_handler/handler.py", line 113, in wrapper\n    for name, data, acl in meth(self, *args, **kwargs):\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/splunktaucclib/rest_handler/handler.py", line 299, in _format_response\n    masked = self.rest_credentials.decrypt_for_get(name, data)\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/splunktaucclib/rest_handler/credentials.py", line 184, in decrypt_for_get\n    clear_password = self._get(name)\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/splunktaucclib/rest_handler/credentials.py", line 389, in _get\n    string = mgr.get_password(user=context.username())\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/solnlib/utils.py", line 154, in wrapper\n    return func(*args, **kwargs)\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/solnlib/credentials.py", line 118, in get_password\n    all_passwords = self._get_all_passwords()\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/solnlib/utils.py", line 154, in wrapper\n    return func(*args, **kwargs)\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/solnlib/credentials.py", line 272, in _get_all_passwords\n    clear_password += field_clear[index]\nTypeError: cannot concatenate 'str' and 'NoneType' objects\n\n
07-19-2017 18:06:36.642 +0800 ERROR AdminManagerExternal - Unexpected error "<class 'splunktaucclib.rest_handler.error.RestError'>" from python handler: "REST Error [500]: Internal Server Error -- Traceback (most recent call last):\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/splunktaucclib/rest_handler/handler.py", line 113, in wrapper\n    for name, data, acl in meth(self, *args, **kwargs):\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/splunktaucclib/rest_handler/handler.py", line 299, in _format_response\n    masked = self.rest_credentials.decrypt_for_get(name, data)\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/splunktaucclib/rest_handler/credentials.py", line 184, in decrypt_for_get\n    clear_password = self._get(name)\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/splunktaucclib/rest_handler/credentials.py", line 389, in _get\n    string = mgr.get_password(user=context.username())\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/solnlib/utils.py", line 154, in wrapper\n    return func(*args, **kwargs)\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/solnlib/credentials.py", line 118, in get_password\n    all_passwords = self._get_all_passwords()\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/solnlib/utils.py", line 154, in wrapper\n    return func(*args, **kwargs)\n  File "/opt/splunk/etc/apps/TA-Cb_Defense/bin/ta_cb_defense/solnlib/credentials.py", line 272, in _get_all_passwords\n    clear_password += field_clear[index]\nTypeError: cannot concatenate 'str' and 'NoneType' objects\n".  See splunkd.log for more details.

Anyone have any ideas what's the issue?

[ZacEsa]

I found out what the issue is already.

I added the inputs and the API key and connect ID through my Splunk master web UI, and then I copied the generated configuration files to the deployment-apps folder to be pushed to my heavy forwarder. However, this would not work as the encryption would be different.
Note: The app encrypts the API key and connector ID.

To solve this issue, I removed the deployed app so that the forwarder will not pull the apps from the master, then I enabled the webserver on the heavy forwarder using ./splunk enable webserver, installed the app directly to the heavy forwarder and configured from the heavy forwarder web UI. After that, I disabled the webserver using ./splunk disable webserver to save resources.