Splunk Python interpreter not recognizing C library module?

[corkster919]

I am currently attempting to build in a new command. This command includes code from the msgspec python package. Within the package is a C library which is referenced as a module by the other functions of the package.

Testing on my Splunk Ubuntu server and base python3 versions 3.6, 3.7, 3.8, 3.9, and 3.10, this package works without issue. When I attempt to run the same package reference from a python script called via the splunk command line, I receive an error.

Given I can successfully run a test script from the same <app>/bin directory as my splunk commands using the same modules, is there anything in Splunk's use of the python interpreter that would prevent Splunk from using Python's built-in C-extension library?

Error code example:

_core  --> reference: https://github.com/jcrist/msgspec/blob/main/msgspec/_core.c

Successfully created new dispatch directory for search job. sid=b762c107adc97090_tmp dispatch_dir=/opt/splunk/var/run/splunk/dispatch/b762c107adc97090_tmp
10-31-2022 13:33:02.556 INFO  ChunkedExternProcessor [782335 searchOrchestrator] - Running process: /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/<app>/bin/<base_script>.py
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr: Traceback (most recent call last):
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/<app>/bin/<base_script>.py", line 14, in <module>
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr:     from <parse_script> import <parse_function> as <function>
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/<app>/bin/lib/<parse_script>.py", line 3, in <module>
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr:     from msgspec.json import encode, decode
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/<app>/bin/lib/msgspec/__init__.py", line 1, in <module>
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr:     from ._core import (
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr: ModuleNotFoundError: No module named 'msgspec._core'
10-31-2022 13:33:02.685 ERROR ChunkedExternProcessor [782335 searchOrchestrator] - EOF while attempting to read transport header read_size=0
10-31-2022 13:33:02.708 ERROR ChunkedExternProcessor [782335 searchOrchestrator] - Error in '<command>' command: External search command exited unexpectedly with non-zero error code 1.

[richgalloway]

Splunk external commands use the Splunk-provided Python interpreter and libraries.  If you need a library not supplied by Splunk then put it in the 'lib' directory of your app.

[corkster919]

It is. It's identified here, in the error reference I attached to the question.

File "/opt/splunk/etc/apps/<app>/bin/lib/<parse_script>.py", line 3, in <module>

[richgalloway]

As the error messages say, msgspec._core is missing.  Add it to the lib directory.

[corkster919]

"Given I can successfully run a test script from the same <app>/bin directory as my splunk commands using the same modules...."

Please note that in my original response, I directly highlight that I run the EXACT SAME MODULE from the EXACT SAME DIRECTORY via terminal and it works. It works on five separate major python release versions. I literally run a test script that does the same thing the splunk script does, just without the splunklib components... and it works without issue. That implication means it's already there. It's also referenced in the error message 

As it states below in the error message included in the original post, it is evident that the module is called directly from the same module directory as the __init__.py file. As is easily evident from this github page containing the python module source code, https://github.com/jcrist/msgspec/tree/main/msgspec, it is clear the _core.c file is downloaded simultaneously with the entire package. 

File "/opt/splunk/etc/apps/<app>/bin/lib/<parse_script>.py", line 3, in <module>
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr:     from msgspec.json import encode, decode
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr:   File "/opt/splunk/etc/apps/<app>/bin/lib/msgspec/__init__.py", line 1, in <module>
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr:     from ._core import (
10-31-2022 13:33:02.674 ERROR ChunkedExternProcessor [782340 ChunkedExternProcessorStderrLogger] - stderr: ModuleNotFoundError: No module named 'msgspec._core'

I will re-ask the same question. Again. Is there anything different with "Splunk's python interpreter" (*note I reference the specific python interpreter used by Splunk) that would prevent C libraries being used as part of the Python built-in C-extension library?

I've written hundreds of custom scripts for Splunk. This is why my question is explicit with lots of context in the OP.

[richgalloway]

I get the impression you've tested the script outside the Splunk run-time environment and it works fine.  The best way to test a Splunk script is within the Splunk environment.

splunk cmd python foo

I understand your frustration.  I'm merely pointing out that the error message says it can't find a needed module.  The fix for that is to put the module in the app.  FWIW, I use etc/<app>/lib rather than etc/<app>/bin/lib for my libraries.