I am currently attempting to build in a new command. This command includes code from the msgspec python package. Within the package is a C library which is referenced as a module by the other functions of the package.
Testing on my Splunk Ubuntu server and base python3 versions 3.6, 3.7, 3.8, 3.9, and 3.10, this package works without issue. When I attempt to run the same package reference from a python script called via the splunk command line, I receive an error. Given I can successfully run a test script from the same <app>/bin directory as my splunk commands using the same modules, is there anything in Splunk's use of the python interpreter that would prevent Splunk from using Python's built-in C-extension library?
Error code example:
_core --> reference: https://github.com/jcrist/msgspec/blob/main/msgspec/_core.c
Successfully created new dispatch directory for search job. sid=b762c107adc97090_tmp dispatch_dir=/opt/splunk/var/run/splunk/dispatch/b762c107adc97090_tmp
10-31-2022 13:33:02.556 INFO ChunkedExternProcessor [782335 searchOrchestrator] - Running process: /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/<app>/bin/<base_script>.py
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr: Traceback (most recent call last):
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr: File "/opt/splunk/etc/apps/<app>/bin/<base_script>.py", line 14, in <module>
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr: from <parse_script> import <parse_function> as <function>
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr: File "/opt/splunk/etc/apps/<app>/bin/lib/<parse_script>.py", line 3, in <module>
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr: from msgspec.json import encode, decode
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr: File "/opt/splunk/etc/apps/<app>/bin/lib/msgspec/__init__.py", line 1, in <module>
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr: from ._core import (
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr: ModuleNotFoundError: No module named 'msgspec._core'
10-31-2022 13:33:02.685 ERROR ChunkedExternProcessor [782335 searchOrchestrator] - EOF while attempting to read transport header read_size=0
10-31-2022 13:33:02.708 ERROR ChunkedExternProcessor [782335 searchOrchestrator] - Error in '<command>' command: External search command exited unexpectedly with non-zero error code 1.
... View more
