topic Re: How to use a different Python Version? in Splunk Dev

How to have Splunk call my custom Python script that is using a different version/installation of Python?

jrouse025 — Wed, 17 Jun 2020 23:11:52 GMT

How can I have Splunk call my custom python script that is using a different version/installation of python?

Requirements:
a. Windows 2012
b. do not want to modify the splunk python 2.7
c. My Python install location d:\mypython
d. Splunk can call script d:\Splunk\etc\apps\search\bin\mycustompassthrough.py
e. search results passed in
f. results passed back to splunk

Thanks for any assistance..cheers.

Re: How to use a different Python Version?

gjanders — Mon, 21 Jan 2019 23:10:33 GMT

Would PyDev help here ? If not you could use a wrapper script in Windows to call your python command and that should work...

Re: How to use a different Python Version?

jrouse025 — Tue, 22 Jan 2019 13:05:28 GMT

Thank you for the suggestion, at first glance it appears it may do the job.

Re: How to use a different Python Version?

jrouse025 — Mon, 01 Apr 2019 15:04:19 GMT

@gjanders thank you for your suggestion. The PyDev application was a good idea to gain some inspiration to move to using python virtual environments (which is exactly what we did).
Here how you can be successful:

Given: custom command running your python script

Prep:
1. Create a python virtual environment (see python documentation site)link text
2. Install needed modules into your python virtual environment
3. Update your python script to use the Activate/activation method (see python documentation)link text
You could also look at how PyDen works for further inspiration
SUCCESS! Our script will now run inside the custom python environment that contains its needed modules without having to modify Splunk.

Re: How to use a different Python Version?

gjanders — Mon, 01 Apr 2019 20:41:02 GMT

@jrouse25 perhaps you can answer your question with the above comment and accept it? Glad you found a solution

Re: How to use a different Python Version?

jrouse025 — Tue, 02 Apr 2019 12:00:03 GMT

Given: custom command running your python script

Prep:
1. Create a python virtual environment (see python documentation site)link text
2. Install needed modules into your python virtual environment
3. Update your python script to use the Activate/activation method (see python documentation)link text
You could also look at how PyDen works for further inspiration
SUCCESS! Our script will now run inside the custom python environment that contains its needed modules without having to modify Splunk.

Re: How to use a different Python Version?

jrouse025 — Tue, 30 Apr 2019 19:26:14 GMT

We've had to move away from Python Virtual environments activate process that Pyden is using (proved to be a very unstable integration process with Splunk)

**But the solution appears to be using Python Subprocess module. Here is a post describing our implementation. link text

Re: How to use a different Python Version?

badarsebard — Fri, 07 Jun 2019 06:32:25 GMT

The particular process I used for PyDen to activate virtual environments has a known bug for Windows. There's a work around available but since PyDen doesn't support Windows currently for other reasons I haven't included it in the app.

Additionally, PyDen does not utilize Intersplunk or any other Splunk libraries because one of its goals was to support Python3 which is incompatible with the internal Splunk libraries.

These, I think, are the two main integration issues probably experienced.

Re: How to use a different Python Version?

hallt3 — Fri, 07 Jun 2019 19:12:09 GMT

Worst case scenario most python 3.x features have been back-ported to python 2.7.x. and you can add any needed libraries to the splunk install if needed.