I have been looking around and I have seen some people having issues with certain dependencies and after dealing with issues related to the usage of manually added modules by dragging them into the aob_py3 folder I have been trying to find information on what the "official and proper" way to have add on builder add in/use additional libraries as needed. I feel like manually moving the needed libs/modules into the aob_py3 folder can't be the best solution but it might be the only one...
I have had some odd solutions to my splunk instance not properly working with modules/libs added in using the above method, and that was..... to install by doing the following.
cd /opt/splunk/bin sudo su splunk ./splunk cmd python #to install packages, in package_names you can add a , and append multiple. import pip; package_names=['grpcio'] ; pip.main(['install'] + package_names + ['--upgrade'])
in other words, going straight to the local python env that splunk uses, acting as splunk, and directly installing the package here.
big problem with this solution.... it can't be packaged up to then be imported using add-on builder. I have tried using the same module shown above, grcp, in a lot of ways but it has some sort of issue with locating or running in general unless I do it as shown above, side note there are many others imported just fine it seems like this one in particular handles things differently as it operates a bit differently compared to other modules.
I know that with docker container you can specify the dependencies to be installed etc and you can work with a nice little config to define these things and I just wanted to reach out on here as the product page specified to do so and see if there is any context I can be given to find a better way to resolve this?
I thought maybe if I install the dependencies the way I show above and then use add on builder to create a new app that maybe it would pack in that lib into that aobe folder to then allow for packaging up but it doesn't work that way.
This post helped me immensely with dealing with import version issues when working with the Splunk Python SDK. In my dev environment, pip install would default to python 3.8, and when I would copy the downloaded module into my project it would error out because my Splunk instance is using Python 3.7.11. By pip installing packages in this manner, I am able to effectively install modules with the piece of mind that they will be the correct version for my Splunk instance.
Installed modules are found in this directory (Ubuntu 20): /opt/splunk/lib/python3.7
Splunk Enterprise Version: 9.0.1
Working with Generating Search Commands and the BigQuery API
===ABOUT THE LIB IMPORT ISSUE=== a more detailed cover on how me and my coworker resolved this issue..
first we tried editing the permissions for files, making them all readable/editable/executable as needed within grpc's folders.
after that failed, we installed the grpc module using python directly within splunks bin.
you could likely do this to generate a version that will work with the app, and export that to where it is used in the integration itself....that is what we did.
sudo su splunk
./splunk cmd python
#to install packages, in package_names you can add a , and append multiple. import pip; package_names=['grpcio'] ; pip.main(['install'] + package_names + ['--upgrade']) #to remove packages, in package_names you can add a , and append multiple. import pip; package_names=['grpcio'] ; pip.main(['uninstall'] + package_names) #to show file location where the package installed was stored. import pip; package_names=['grpcio'] ; pip.main(['show'] + package_names) #to check that your using 64bit python. import platform; platform.architecture()
you could edit the above line to uninstall it the same way you installed it.
I'd like to purge my aob_py3 folder as much as possible so that I can put my add-on into source control. Browsing around GitHub, I see a wide range of aob_py3 contents. Some examples:
as someone gave kudos, I wanted to update this post.
if you do the above, copy the installed lib from the main python environment running splunk, and then place that into the add-on builder bin dir for libs, and properly change the ownership and access rights, I had success. this is assuming anyone is having a issue with the same import.
I could then package it and import it to a system that had never performed the prior steps and it worked. Likely a issue with the version installed, or the binary created, or some other component unsure.
I still would like a answer to my original post however if possible.