Developing for Splunk Platform

How can I add a Python modules to the Splunk Django environment?

heshamzaid
Explorer

I need to add a Python module that is not included in the Splunk python. I want to use this module in Django framework, but the answers in this previous question do not satisfy my requirement:
https://answers.splunk.com/answers/8/can-i-add-python-modules-to-the-splunk-environment.html

any help

Labels (2)
1 Solution

adambloom
Path Finder

To add numpy, scipy, pandas, scikit-learn, or statsmodels, there's an app on Splunkbase you can install - Python for Scientific Computing. You need to use the correct version for your OS, as many of these libraries have OS specific dependencies. Here's the Linux 64 bit link: https://splunkbase.splunk.com/app/2882/. The README in the app contains instructions on how to import the libraries into other scripts once that app is installed.

View solution in original post

0 Karma

cod3bee
New Member
  1. Go to your local python folder apart from Splunk and copy the scripts related to pip (e.g. pip-install, easy-pip etc.)
  2. Paste all these scripts in Splunk site-packages “C:\Program Files\Splunk\Python-2.7\Lib\site-packages”
  3. Then open command prompt as an administrator and cd to this location “C:\Program Files\Splunk\Python-2.7\Lib\site-packages” and run pip install module name -t ” C:\Program Files\Splunk\Python-2.7\Lib\site-packages”
  4. Then the specified library/module will start working in the Splunk
0 Karma

joelurtubia
Explorer

Hello, I was the same problem with Mysql module that I was install on my Centos server

Splunk didn't work with this library, because splunk has they own python library...then you can fix it only added on the begin your script all libraries of python and also you must to add the python Centos library too... as this way

Find python packages

[root@xxxx]#find / -name site-packages
/usr/lib/python2.7/site-packages
/usr/lib64/python2.7/site-packages
/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/lib/python2.7/site-packages
/opt/splunk/lib/python2.7/site-packages

Find python binary

[root@xxxx]# whereis python
python: /usr/bin/python2.7 /usr/bin/python /usr/lib/python2.7 /usr/lib64/python2.7 /etc/python /usr/include/python2.7 /opt/splunk/bin/python /opt/splunk/bin/python2.7 /usr/share/man/man1/python.1.gz

include all at begin your script

import sys
sys.path.append('/usr/bin/python2.7')
sys.path.append('/usr/lib/python2.7/site-packages')
sys.path.append('/usr/lib64/python2.7/site-packages')

And that's it , you can run mysql module without any problem and create your alerts with this module.

Mysql Connection

import mysql.connector

I hope that this fix will help you
Joel Urtubia Ugarte

0 Karma

adambloom
Path Finder

To add numpy, scipy, pandas, scikit-learn, or statsmodels, there's an app on Splunkbase you can install - Python for Scientific Computing. You need to use the correct version for your OS, as many of these libraries have OS specific dependencies. Here's the Linux 64 bit link: https://splunkbase.splunk.com/app/2882/. The README in the app contains instructions on how to import the libraries into other scripts once that app is installed.

0 Karma

skhadem
Loves-to-Learn

What if there are other python3 modules I want to add?  Such as openssl, websockets?  Other modules outside of scientific computing?  I'm building a TA that is a wrapper for an existing python3 REST api for one of our products, I don't want to ask users to point to an alternative python3 path and script API calls to come from there, but that seems like the only choice, is there any easier way to just specify the modules I want Splunk python3 to use?

0 Karma

jreacher21
New Member

This has been awhile since there was a response to this. I am having similar problems trying to import pandas into a python script. Is there anyway we can import modules to the splunk python script?

Using the Python for Scientific Computing add-on gives me a

Traceback (most recent call last):
File "PackageCmsResults.py", line 15, in <module>
import exec_anaconda
File "/opt/splunk/etc/apps/XXXXXXXX/bin/exec_anaconda.py", line 17, in <module>
from util.base_util import get_apps_path
ModuleNotFoundError: No module named 'util'

 

0 Karma

heshamzaid
Explorer

thanks for your answer .
do you have any idea for matlab module

0 Karma

adambloom
Path Finder

You have a few options:

  • simply add the library locally to the bin folder of your app that will hold your scripted input (or $SPLUNK_HOME/bin/scripts if a script). Local imports will always work
  • If it needs to be global, add to $SPLUNK_HOME/lib/python2.7/site-packages, although that might be harder/less friendly to keep up to date between upgrades/different environments.

My recommendation would be the first option. All you need to do is place your library in the same folder, and then you can simply call import from your script. import always searches the local folder in addition to any system paths.

0 Karma

heshamzaid
Explorer

i want to add numpy module to python in splunk ,
i follow the frist option and get the /usr/lib/python2.7/dist-package/numpy
to the bin folder of my project but it still not work .

0 Karma

adambloom
Path Finder

Ah...numpy is a little more intricate. New answer forthcoming

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...