How to have Splunk call my custom Python script that is using a different version/installation of Python?

[jrouse025]

How can I have Splunk call my custom python script that is using a different version/installation of python?

Requirements:
a. Windows 2012
b. do not want to modify the splunk python 2.7
c. My Python install location d:\mypython
d. Splunk can call script d:\Splunk\etc\apps\search\bin\mycustompassthrough.py
e. search results passed in
f. results passed back to splunk

Thanks for any assistance..cheers.

[hallt3]

Worst case scenario most python 3.x features have been back-ported to python 2.7.x. and you can add any needed libraries to the splunk install if needed.

[jrouse025 · Accepted solution]

@gjanders thank you for your suggestion. The PyDev application was a good idea to gain some inspiration to move to using python virtual environments (which is exactly what we did).
Here how you can be successful:

Given: custom command running your python script

Prep:
1. Create a python virtual environment (see python documentation site)link text
2. Install needed modules into your python virtual environment
3. Update your python script to use the Activate/activation method (see python documentation)link text
You could also look at how PyDen works for further inspiration
SUCCESS! Our script will now run inside the custom python environment that contains its needed modules without having to modify Splunk.

[jrouse025]

We've had to move away from Python Virtual environments activate process that Pyden is using (proved to be a very unstable integration process with Splunk)

**But the solution appears to be using Python Subprocess module. Here is a post describing our implementation. link text

[badarsebard]

The particular process I used for PyDen to activate virtual environments has a known bug for Windows. There's a work around available but since PyDen doesn't support Windows currently for other reasons I haven't included it in the app.

Additionally, PyDen does not utilize Intersplunk or any other Splunk libraries because one of its goals was to support Python3 which is incompatible with the internal Splunk libraries.

These, I think, are the two main integration issues probably experienced.

[gjanders]

Would PyDev help here ? If not you could use a wrapper script in Windows to call your python command and that should work...

[jrouse025]

Thank you for the suggestion, at first glance it appears it may do the job.

[jrouse025]

@gjanders thank you for your suggestion. The PyDev application was a good idea to gain some inspiration to move to using python virtual environments (which is exactly what we did).
Here how you can be successful:

Given: custom command running your python script

Prep:
1. Create a python virtual environment (see python documentation site)link text
2. Install needed modules into your python virtual environment
3. Update your python script to use the Activate/activation method (see python documentation)link text
You could also look at how PyDen works for further inspiration
SUCCESS! Our script will now run inside the custom python environment that contains its needed modules without having to modify Splunk.

[gjanders]

@jrouse25 perhaps you can answer your question with the above comment and accept it? Glad you found a solution