Input tab doesn't load : Microsoft Azure Add-On

[venkattm]

Hi,

We have to ingest activity log into Splunk. We installed Microsoft add-on for Splunk on our heavy forwarder. When we click on the input tab it keeps spinning and I see a bunch of exceptions in the log. I tested installing on a test box with same network config, everything works fine but on our heavy forwarder is an issue. OS is centos box. Any Azure adds on the same issue. I figured out some configuration its not able to read but what it is is something I am not able to figure out.

6-30-2020 15:02:50.557 -0400 DEBUG AdminManagerExternal - Sending handler setup data:\n<?xml version="1.0" encoding="UTF-8"?>\n<eai>\n <eai_settings>\n <appName>TA-MS-AAD</appName>\n <userName>nobody</userName>\n <customAction></customAction>\n <maxCount>0</maxCount>\n <posOffset>0</posOffset>\n <requestedAction>2</requestedAction>\n <shouldFilter></shouldFilter>\n <sortAscending>true</sortAscending>\n <sortByKey>name</sortByKey>\n </eai_settings>\n <sessionKey>vIhbfLFoBcWd2QCagd7Gg3k6kN0gvxARFwcRvaUefynzDvBAEtTKQGjoqELPNBmQNNcZ^rQ7NhjTKMMgA^9h^M3w41LuqWnWKO4XMQA_P7uzDxlio1cfOHdHoBp7w3MNy1voyC</sessionKey>\n <productType>enterprise</productType>\n <callerArgs>\n <id></id>\n <args/>\n </callerArgs>\n <setup/>\n</eai>\n
06-30-2020 15:02:50.557 -0400 DEBUG AdminManagerExternal - Sending handler setup data:\n<?xml version="1.0" encoding="UTF-8"?>\n<eai>\n <eai_settings>\n <appName>TA-MS-AAD</appName>\n <userName>nobody</userName>\n <customAction></customAction>\n <maxCount>0</maxCount>\n <posOffset>0</posOffset>\n <requestedAction>2</requestedAction>\n <shouldFilter></shouldFilter>\n <sortAscending>true</sortAscending>\n <sortByKey>name</sortByKey>\n </eai_settings>\n <sessionKey>vIhbfLFoBcWd2QCagd7Gg3k6kN0gvxARFwcRvaUefynzDvBAEtTKQGjoqELPNBmQNNcZ^rQ7NhjTKMMgA^9h^M3w41LuqWnWKO4XMQA_P7uzDxlio1cfOHdHoBp7w3MNy1voyC</sessionKey>\n <productType>enterprise</productType>\n <callerArgs>\n <id></id>\n <args/>\n </callerArgs>\n <setup/>\n</eai>\n
06-30-2020 15:02:50.558 -0400 DEBUG AdminManagerExternal - Sending handler setup data:\n<?xml version="1.0" encoding="UTF-8"?>\n<eai>\n <eai_settings>\n <appName>TA-MS-AAD</appName>\n <userName>nobody</userName>\n <customAction></customAction>\n <maxCount>0</maxCount>\n <posOffset>0</posOffset>\n <requestedAction>2</requestedAction>\n <shouldFilter></shouldFilter>\n <sortAscending>true</sortAscending>\n <sortByKey>name</sortByKey>\n </eai_settings>\n <sessionKey>vIhbfLFoBcWd2QCagd7Gg3k6kN0gvxARFwcRvaUefynzDvBAEtTKQGjoqELPNBmQNNcZ^rQ7NhjTKMMgA^9h^M3w41LuqWnWKO4XMQA_P7uzDxlio1cfOHdHoBp7w3MNy1voyC</sessionKey>\n <productType>enterprise</productType>\n <callerArgs>\n <id></id>\n <args/>\n </callerArgs>\n <setup/>\n</eai>\n
06-30-2020 15:02:50.563 -0400 DEBUG AdminManagerExternal - Got back data: <eai_error><recognized>false</recognized><type>&lt;class 'backports.configparser.InterpolationSyntaxError'&gt;</type><message>'%' must be followed by '%' or '(', found: '%8x8O'</message><stacktrace>Traceback (most recent call last):\n File "/opt/splunk/lib/python3.7/site-packages/splunk/admin.py", line 151, in init\n hand = handler(mode, ctxInfo)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/splunktaucclib/rest_handler/admin_external.py", line 67, in __init__\n get_splunkd_uri(),\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 210, in get_splunkd_uri\n scheme, host, port = get_splunkd_access_info()\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 182, in get_splunkd_access_info\n 'server', 'sslConfig', 'enableSplunkdSSL')):\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 230, in get_conf_key_value\n stanzas = get_conf_stanzas(conf_name)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 284, in get_conf_stanzas\n out[section] = {item[0]: item[1] for item in parser.items(section)}\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 870, in items\n return [(option, value_getter(option)) for option in d.keys()]\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 870, in &lt;listcomp&gt;\n return [(option, value_getter(option)) for option in d.keys()]\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 867, in &lt;lambda&gt;\n section, option, d[option], d)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 387, in before_get\n self._interpolate_some(parser, option, L, value, section, defaults, 1)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 437, in _interpolate_some\n "found: %r" % (rest,))\nbackports.configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%8x8O'\n</stacktrace></eai_error>\n
06-30-2020 15:02:50.563 -0400 DEBUG AdminManagerExternal - Found serialized error from external handler.
06-30-2020 15:02:50.563 -0400 ERROR AdminManagerExternal - Stack trace from python handler:\nTraceback (most recent call last):\n File "/opt/splunk/lib/python3.7/site-packages/splunk/admin.py", line 151, in init\n hand = handler(mode, ctxInfo)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/splunktaucclib/rest_handler/admin_external.py", line 67, in __init__\n get_splunkd_uri(),\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 210, in get_splunkd_uri\n scheme, host, port = get_splunkd_access_info()\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 182, in get_splunkd_access_info\n 'server', 'sslConfig', 'enableSplunkdSSL')):\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 230, in get_conf_key_value\n stanzas = get_conf_stanzas(conf_name)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 284, in get_conf_stanzas\n out[section] = {item[0]: item[1] for item in parser.items(section)}\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 870, in items\n return [(option, value_getter(option)) for option in d.keys()]\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 870, in <listcomp>\n return [(option, value_getter(option)) for option in d.keys()]\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 867, in <lambda>\n section, option, d[option], d)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 387, in before_get\n self._interpolate_some(parser, option, L, value, section, defaults, 1)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 437, in _interpolate_some\n "found: %r" % (rest,))\nbackports.configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%8x8O'\n
06-30-2020 15:02:50.563 -0400 ERROR AdminManagerExternal - Unexpected error "<class 'backports.configparser.InterpolationSyntaxError'>" from python handler: "'%' must be followed by '%' or '(', found: '%8x8O'". See splunkd.log for more details.
06-30-2020 15:02:50.569 -0400 DEBUG AdminManagerExternal - Got back data: <eai_error><recognized>false</recognized><type>&lt;class 'backports.configparser.InterpolationSyntaxError'&gt;</type><message>'%' must be followed by '%' or '(', found: '%8x8O'</message><stacktrace>Traceback (most recent call last):\n File "/opt/splunk/lib/python3.7/site-packages/splunk/admin.py", line 151, in init\n hand = handler(mode, ctxInfo)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/splunktaucclib/rest_handler/admin_external.py", line 67, in __init__\n get_splunkd_uri(),\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 210, in get_splunkd_uri\n scheme, host, port = get_splunkd_access_info()\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 182, in get_splunkd_access_info\n 'server', 'sslConfig', 'enableSplunkdSSL')):\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 230, in get_conf_key_value\n stanzas = get_conf_stanzas(conf_name)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 284, in get_conf_stanzas\n out[section] = {item[0]: item[1] for item in parser.items(section)}\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 870, in items\n return [(option, value_getter(option)) for option in d.keys()]\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 870, in &lt;listcomp&gt;\n return [(option, value_getter(option)) for option in d.keys()]\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 867, in &lt;lambda&gt;\n section, option, d[option], d)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 387, in before_get\n self._interpolate_some(parser, option, L, value, section, defaults, 1)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 437, in _interpolate_some\n "found: %r" % (rest,))\nbackports.configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%8x8O'\n</stacktrace></eai_error>\n
06-30-2020 15:02:50.569 -0400 DEBUG AdminManagerExternal - Found serialized error from external handler.
06-30-2020 15:02:50.569 -0400 ERROR AdminManagerExternal - Stack trace from python handler:\nTraceback (most recent call last):\n File "/opt/splunk/lib/python3.7/site-packages/splunk/admin.py", line 151, in init\n hand = handler(mode, ctxInfo)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/splunktaucclib/rest_handler/admin_external.py", line 67, in __init__\n get_splunkd_uri(),\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 210, in get_splunkd_uri\n scheme, host, port = get_splunkd_access_info()\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 182, in get_splunkd_access_info\n 'server', 'sslConfig', 'enableSplunkdSSL')):\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 230, in get_conf_key_value\n stanzas = get_conf_stanzas(conf_name)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/solnlib/splunkenv.py", line 284, in get_conf_stanzas\n out[section] = {item[0]: item[1] for item in parser.items(section)}\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 870, in items\n return [(option, value_getter(option)) for option in d.keys()]\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 870, in <listcomp>\n return [(option, value_getter(option)) for option in d.keys()]\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 867, in <lambda>\n section, option, d[option], d)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 387, in before_get\n self._interpolate_some(parser, option, L, value, section, defaults, 1)\n File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/aob_py3/backports/configparser/__init__.py", line 437, in _interpolate_some\n "found: %r" % (rest,))\nbackports.configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%8x8O'\n
06-30-2020 15:02:50.569 -0400 ERROR AdminManagerExternal - Unexpected error "<class 'backports.configparser.InterpolationSyntaxError'>" from python handler: "'%' must be followed by '%' or '(', found: '%8x8O'". See splunkd.log for more details.

[Rams01]

You ever get this figure out by chance?