Developing for Splunk Enterprise

How to have Splunk call my custom Python script that is using a different version/installation of Python?

jrouse025
Path Finder

How can I have Splunk call my custom python script that is using a different version/installation of python?

Requirements:
a. Windows 2012
b. do not want to modify the splunk python 2.7
c. My Python install location d:\mypython
d. Splunk can call script d:\Splunk\etc\apps\search\bin\mycustompassthrough.py
e. search results passed in
f. results passed back to splunk

Thanks for any assistance..cheers.

Labels (2)
0 Karma
1 Solution

jrouse025
Path Finder

@gjanders thank you for your suggestion. The PyDev application was a good idea to gain some inspiration to move to using python virtual environments (which is exactly what we did).
Here how you can be successful:

Given: custom command running your python script

Prep:
1. Create a python virtual environment (see python documentation site)link text
2. Install needed modules into your python virtual environment
3. Update your python script to use the Activate/activation method (see python documentation)link text
You could also look at how PyDen works for further inspiration
SUCCESS! Our script will now run inside the custom python environment that contains its needed modules without having to modify Splunk.

View solution in original post

hallt3
Path Finder

Worst case scenario most python 3.x features have been back-ported to python 2.7.x. and you can add any needed libraries to the splunk install if needed.

0 Karma

jrouse025
Path Finder

@gjanders thank you for your suggestion. The PyDev application was a good idea to gain some inspiration to move to using python virtual environments (which is exactly what we did).
Here how you can be successful:

Given: custom command running your python script

Prep:
1. Create a python virtual environment (see python documentation site)link text
2. Install needed modules into your python virtual environment
3. Update your python script to use the Activate/activation method (see python documentation)link text
You could also look at how PyDen works for further inspiration
SUCCESS! Our script will now run inside the custom python environment that contains its needed modules without having to modify Splunk.

View solution in original post

jrouse025
Path Finder

We've had to move away from Python Virtual environments activate process that Pyden is using (proved to be a very unstable integration process with Splunk)

**But the solution appears to be using Python Subprocess module. Here is a post describing our implementation. link text

0 Karma

badarsebard
Communicator

The particular process I used for PyDen to activate virtual environments has a known bug for Windows. There's a work around available but since PyDen doesn't support Windows currently for other reasons I haven't included it in the app.

Additionally, PyDen does not utilize Intersplunk or any other Splunk libraries because one of its goals was to support Python3 which is incompatible with the internal Splunk libraries.

These, I think, are the two main integration issues probably experienced.

0 Karma

gjanders
SplunkTrust
SplunkTrust

Would PyDev help here ? If not you could use a wrapper script in Windows to call your python command and that should work...

Alerts for Splunk Admins https://splunkbase.splunk.com/app/3796/
Version Control for Splunk https://splunkbase.splunk.com/app/4355/
0 Karma

jrouse025
Path Finder

Thank you for the suggestion, at first glance it appears it may do the job.

0 Karma

jrouse025
Path Finder

@gjanders thank you for your suggestion. The PyDev application was a good idea to gain some inspiration to move to using python virtual environments (which is exactly what we did).
Here how you can be successful:

Given: custom command running your python script

Prep:
1. Create a python virtual environment (see python documentation site)link text
2. Install needed modules into your python virtual environment
3. Update your python script to use the Activate/activation method (see python documentation)link text
You could also look at how PyDen works for further inspiration
SUCCESS! Our script will now run inside the custom python environment that contains its needed modules without having to modify Splunk.

gjanders
SplunkTrust
SplunkTrust

@jrouse25 perhaps you can answer your question with the above comment and accept it? Glad you found a solution

Alerts for Splunk Admins https://splunkbase.splunk.com/app/3796/
Version Control for Splunk https://splunkbase.splunk.com/app/4355/
0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!