Custom search command in Python and Splunk v8.1

[TaZe78_Fr]

Hello,

Integrated into our app, we had developed a custom search command (streaming) based on Splunk Enterprise SDK for Python v1.6.2 and using Python for Scientific Computing (for Linux 64-bit) v1.4. This command worked fine under Splunk v7.2.9.1.
We are trying to migrate to Splunk v8.1 (latest version available). To accomplish this, we are also migrating Splunk Enterprise SDK for Python to v1.6.15 (latest version available) and Python for Scientific Computing (for Linux 64-bit) to v2.0.2 (latest version available).

The problem is that our command no longer works ... I looked at what the Splunk Platform Upgrade Readiness App indicates, and it only indicates warnings (by the way: 1 on the script of our custom command, 1 on the exec_anaconda.py of Python for Scientific Computing (for Linux 64-bit) and 13 on the Splunk Enterprise SDK for Python).

Here is the error that appears (search.log when executing our custom command):

02-18-2021 18:08:01.794 INFO  ChunkedExternProcessor - Running process: /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/talios/bin/HUMS.py

02-18-2021 18:08:01.865 ERROR ChunkedExternProcessor - stderr: Traceback (most recent call last):

02-18-2021 18:08:01.865 ERROR ChunkedExternProcessor - stderr:   File "/opt/splunk/etc/apps/talios/bin/HUMS.py", line 7, in <module>

02-18-2021 18:08:01.865 ERROR ChunkedExternProcessor - stderr:     import exec_anaconda

02-18-2021 18:08:01.865 ERROR ChunkedExternProcessor - stderr:   File "/opt/splunk/etc/apps/talios/bin/exec_anaconda.py", line 17, in <module>

02-18-2021 18:08:01.865 ERROR ChunkedExternProcessor - stderr:     from util.base_util import get_apps_path

02-18-2021 18:08:01.865 ERROR ChunkedExternProcessor - stderr: ModuleNotFoundError: No module named 'util'

02-18-2021 18:08:01.870 ERROR ChunkedExternProcessor - EOF while attempting to read transport header read_size=0

02-18-2021 18:08:01.870 ERROR ChunkedExternProcessor - Error in 'hums' command: External search command exited unexpectedly with non-zero error code 1.

02-18-2021 18:08:01.872 ERROR SearchPhaseGenerator - Fallback to two phase search failed:Error in 'hums' command: External search command exited unexpectedly with non-zero error code 1.

02-18-2021 18:08:01.873 ERROR SearchOrchestrator - Error in 'hums' command: External search command exited unexpectedly with non-zero error code 1.

02-18-2021 18:08:01.873 ERROR SearchStatusEnforcer - sid:scheduler__admin__talios__RMD5b64ee28fa5a2b66b_at_1613671680_163 Error in 'hums' command: External search command exited unexpectedly with non-zero error code 1.

This is an error in the exec_anaconda.py of Python for Scientific Computing (for Linux 64-bit) ...

By following the solution of this post (https://community.splunk.com/t5/All-Apps-and-Add-ons/How-to-import-util-when-using-exec-anaconda/mp/...) , we then come across the following error:

02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr: Traceback (most recent call last):
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:   File "/opt/splunk/etc/apps/talios/bin/HUMS.py", line 12, in <module>
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:     from splunklib.searchcommands import dispatch, StreamingCommand, Configuration, Option, validators
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:   File "/opt/splunk/etc/apps/talios/bin/splunklib/searchcommands/__init__.py", line 145, in <module>
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:     from .environment import *
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:   File "/opt/splunk/etc/apps/talios/bin/splunklib/searchcommands/environment.py", line 120, in <module>
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:     splunklib_logger, logging_configuration = configure_logging('splunklib')
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:   File "/opt/splunk/etc/apps/talios/bin/splunklib/searchcommands/environment.py", line 103, in configure_logging
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:     fileConfig(filename, {'SPLUNK_HOME': splunk_home})
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/lib/python3.7/logging/config.py", line 80, in fileConfig
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:     _install_loggers(cp, handlers, disable_existing_loggers)
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/lib/python3.7/logging/config.py", line 196, in _install_loggers
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:     log.setLevel(level)
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/lib/python3.7/logging/__init__.py", line 1353, in setLevel
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:     self.level = _checkLevel(level)
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:   File "/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/lib/python3.7/logging/__init__.py", line 192, in _checkLevel
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr:     raise ValueError("Unknown level: %r" % level)
02-19-2021 12:53:48.296 ERROR ChunkedExternProcessor - stderr: ValueError: Unknown level: 'WARNING   ; Default: WARNING'

This is an error in the Splunk Enterprise SDK for Python ...

All this comes from our development environment which is in Docker, based on the official Splunk image (splunk / splunk: 8.1).

If you have any leads see a solution.

Thank you in advance for your help.