Splunk python SDK cannot be used in a TA??

[Alex_S]

I've developed TA's previously, and when using python2, everything worked just fine. But now, using python3 with splunk 9.x, it seems nothing works. Trying to develop a TA that makes some REST calls out to a 3rd-party service, and then uses those values in some local confs. It's been a nightmare to try to make this work.

Started with a modular input design, but contrary to the docs, my python code would never receive a splunk token on STDIN. Literally had this working perfectly in a python2 TA. This time? Doesn't matter how or when attempting to read STDIN, the python3 code *NEVER RECEIVES ANYTHING*. Finally I just gave up on this...

Next try was with a scripted input; at least this **bleep** thing does receive a token on STDIN. Great, that token can be used w/ the SDK, right?  RIGHT???

Well, no, because 1) splunklib is not installed/included in the splunk python env, and 2) attempting to use the system python causes the whole **bleep** thing to crash, and 3) including splunklib inside the TA, and attempting to import it by manipulating python paths is also horribly broken.

If we munge the python system paths thusly, we can in theory import our included libs (not concerned if this is idiomatic python; it works m'kay?):

import os, sys

modules = sys.argv[0].split('/')[:-2]

modules.append('lib')

sys.path.append('/'.join(modules))

This inserts our local lib path into python's lib search dirs. And it works to find splunklib. But then splunklib fails to load completely since:

ImportError: libssl.so.1.0.0: cannot open shared object file: No such file or directory

This is true even if LD_LIBRARY_PATH points to a dir containing libssl.so.1.0.0. I suspect this is due to the fact that Splunk is also doing an LD_PRELOAD="libdlwrapper.so"

I don't know what this library is or what it's doing, but I also suspect it's breaking my env preventing anything from running. But it doesn't actually matter.

If I remove my "import splunklib" and just leave the REST client to attempt to make its HTTPS request, that too is apparently horribly broken:

...(Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available"))

What in the everloving fsck is going on here??!? Best I can tell, these two things are now true:

1) splunklib cannot be used from a TA

2) TA's cannot make HTTPS requests

This is happening in a clean-room environment with a fresh splunk install on a host that is not running selinux or apparmor or any other MAC system that might interfere. This is very much a problem with Splunk and splunklib. 

So, how exactly can splunklib be used in TAs? And how exactly can TAs execute HTTPS requests??

[livehybrid]

Hi @Alex_S 

splunklib shouldnt be an issue in TAs - I use this all the time. How are you testing/executing your script? Ive only ever seen this issue when running using the system python rather than Splunks python build.

I'd highly recommend checking out https://conf.splunk.com/files/2024/slides/DEV1885B.pdf and getting to know UCC which makes it very easy to start building modular inputs. The boilerplate code is pretty much ready to go, just stick in your requests and off you go!

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing