All Apps and Add-ons

Palo Alto Networks Add-on error with rest_migration.py

matthewroberson
Path Finder

I updated my Palo Alto Networks Add-on to version 6.3.1 and now I'm seeing the errors below in splunkd.log on the search head cluster members the add-on is deployed to.

 

 

 

09-03-2020 09:54:10.323 -0500 ERROR AdminManagerExternal - Stack trace from python handler:\nTraceback (most recent call last):\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 19, in handle\n    return func(*args, **kwargs)\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 71, in _migrate\n    self._migrate_conf_credential()\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 161, in _migrate_conf_credential\n    conf_file, stanzas = self._load_conf(conf_file_name)\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 178, in _load_conf\n    stanzas = conf_file.get_all()\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/solnlib/utils.py", line 159, in wrapper\n    return func(*args, **kwargs)\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/solnlib/conf_manager.py", line 241, in get_all\n    key_values = self._decrypt_stanza(name, stanza_mgr.content)\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/solnlib/conf_manager.py", line 126, in _decrypt_stanza\n    self._cred_mgr.get_password(stanza_name))\n  File "/opt/splunk/lib/python3.7/json/__init__.py", line 348, in loads\n    return _default_decoder.decode(s)\n  File "/opt/splunk/lib/python3.7/json/decoder.py", line 337, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n  File "/opt/splunk/lib/python3.7/json/decoder.py", line 355, in raw_decode\n    raise JSONDecodeError("Expecting value", s, err.value) from None\njson.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File "/opt/splunk/lib/python3.7/site-packages/splunk/admin.py", line 148, in init\n    hand.execute(info)\n  File "/opt/splunk/lib/python3.7/site-packages/splunk/admin.py", line 634, in execute\n    if self.requestedAction == ACTION_LIST:     self.handleList(confInfo)\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 36, in handleList\n    self._migrate()\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 23, in handle\n    'Migrating failed. %s' % traceback.format_exc()\nsplunktaucclib.rest_handler.error.RestError: REST Error [500]: Internal Server Error -- Migrating failed. Traceback (most recent call last):\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 19, in handle\n    return func(*args, **kwargs)\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 71, in _migrate\n    self._migrate_conf_credential()\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 161, in _migrate_conf_credential\n    conf_file, stanzas = self._load_conf(conf_file_name)\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 178, in _load_conf\n    stanzas = conf_file.get_all()\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/solnlib/utils.py", line 159, in wrapper\n    return func(*args, **kwargs)\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/solnlib/conf_manager.py", line 241, in get_all\n    key_values = self._decrypt_stanza(name, stanza_mgr.content)\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/solnlib/conf_manager.py", line 126, in _decrypt_stanza\n    self._cred_mgr.get_password(stanza_name))\n  File "/opt/splunk/lib/python3.7/json/__init__.py", line 348, in loads\n    return _default_decoder.decode(s)\n  File "/opt/splunk/lib/python3.7/json/decoder.py", line 337, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n  File "/opt/splunk/lib/python3.7/json/decoder.py", line 355, in raw_decode\n    raise JSONDecodeError("Expecting value", s, err.value) from None\njson.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)\n\n
09-03-2020 09:54:10.323 -0500 ERROR AdminManagerExternal - Unexpected error "<class 'splunktaucclib.rest_handler.error.RestError'>" from python handler: "REST Error [500]: Internal Server Error -- Migrating failed. Traceback (most recent call last):\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 19, in handle\n    return func(*args, **kwargs)\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 71, in _migrate\n    self._migrate_conf_credential()\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 161, in _migrate_conf_credential\n    conf_file, stanzas = self._load_conf(conf_file_name)\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py", line 178, in _load_conf\n    stanzas = conf_file.get_all()\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/solnlib/utils.py", line 159, in wrapper\n    return func(*args, **kwargs)\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/solnlib/conf_manager.py", line 241, in get_all\n    key_values = self._decrypt_stanza(name, stanza_mgr.content)\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/solnlib/conf_manager.py", line 126, in _decrypt_stanza\n    self._cred_mgr.get_password(stanza_name))\n  File "/opt/splunk/lib/python3.7/json/__init__.py", line 348, in loads\n    return _default_decoder.decode(s)\n  File "/opt/splunk/lib/python3.7/json/decoder.py", line 337, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n  File "/opt/splunk/lib/python3.7/json/decoder.py", line 355, in raw_decode\n    raise JSONDecodeError("Expecting value", s, err.value) from None\njson.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)\n".  See splunkd.log for more details.

 

 

I took a look at rest_migration.py and it looks to me like it's looking for credentials from an older version of the TA that wasn't installed on my search heads (I'm not great with python, so I could be wrong). The add-on is deployed to a 4 member search head cluster with my deployer. Anyone have any ideas on how to resolve this? As it is, when i try to configure accounts or add-on settings in the app i just get a spinning wheel that says loading.

Labels (1)
0 Karma
1 Solution

matthewroberson
Path Finder

I resolved the issue by removing app and add-on from /opt/splunk/etc/shcluster/apps on the deployer and deploying ( to remove them from the search head ). Then I put the app and add-on back in /opt/splunk/etc/shcluster/apps and redeployed. This resolved the issue.

View solution in original post

0 Karma

matthewroberson
Path Finder

I resolved the issue by removing app and add-on from /opt/splunk/etc/shcluster/apps on the deployer and deploying ( to remove them from the search head ). Then I put the app and add-on back in /opt/splunk/etc/shcluster/apps and redeployed. This resolved the issue.

0 Karma

thambisetty
SplunkTrust
SplunkTrust

What credentials are you talking about?

are you making use of alert actions to manage your firewall from splunk TA?

————————————
If this helps, give a like below.
0 Karma

matthewroberson
Path Finder
    def get_legacy_passwords(self):
        if self.legacy_passwords is None:
            self.legacy_passwords = {}
            for pwd in self.client.storage_passwords.list(count=-1):
                if pwd.realm == self.base_app_name:
                    self.legacy_passwords[pwd.username] = pwd
        return self.legacy_passwords

The above is what I noticed and also the snippet from the error messages below:

self._migrate_conf_credential()\n  File "/opt/splunk/etc/apps/Splunk_TA_paloalto/bin/splunk_ta_paloalto/aob_py3/splunk_aoblib/rest_migration.py

 

I don't have the best grasp of Python, so maybe I'm way off base?

0 Karma

matthewroberson
Path Finder

I am not making use of alert actions to manage your firewall from splunk TA. Intended to include that in my original reply...

0 Karma

thambisetty
SplunkTrust
SplunkTrust

what is your splunk enterprise version ?

https://community.splunk.com/t5/All-Apps-and-Add-ons/Palo-Alto-Networks-Add-on-6-3-1-upgrade-on-Splu...

————————————
If this helps, give a like below.
0 Karma

matthewroberson
Path Finder

8.0.5 running on linux servers...

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...