I have a python script that pulls data from an SFTP source and writes the data to a file (myScript.py). The script imports two modules that I installed. As such, I wrote a bash wrapper script to call my script and run it with standard python (myScriptLauncher.sh).
I can run the bash script from the command line fine. However, when I try to run it using Splunk (./splunk cmd /opt/splunk/etc/apps/myApp/bin/myScriptLauncher.sh), I get a module import error.
I'm not sure what I am doing wrong. Anyone have any suggestions?
error
Traceback (most recent call last):
File "./myScript.py", line 4, in <module>
import pysftp
ImportError: No module named pysftp
myScript.py
#!/usr/bin/env python
import json
import pysftp
import openpyxl
REST OF CODE GOES HERE
myScriptLauncher.sh
#!/bin/bash
echo myScript is starting >&2
cd $( dirname "${BASH_SOURCE[0]}" )
exec /usr/bin/python ./myScript.py
Creating a wrapper to call a Python installation on the host has been the conventional Splunk wisdom for years, as was bundling needed modules into the app. Both these approaches have numerous problems though. Using the wrapper is typically temperamental to permissions, environment variables, and in the case of the default system Python possibly versions of Python even older than 2.7.
My suggestion is to check out a new app I've written called PyDen which allows you to create Python virtual environments and install PyPI packages to the environment. You can then use these environments to execute custom commands and scripted inputs.
This provides three key advantages:
@nakiamatthews
I think you have installedpysftp
files in local python. So you should add path in your myScript.py
script OR you have to copy those file into splunk lib.
See below ref link for your more info:
https://answers.splunk.com/answers/222934/modular-input-that-need-additional-python-modules.html
https://answers.splunk.com/answers/220196/import-non-native-python-libraries-into-splunk.html#answer...
https://answers.splunk.com/answers/8/can-i-add-python-modules-to-the-splunk-environment.html
I was under the impression that adding the modules to the splunk lib was not future proof - i.e. it will break whenever you upgrade Splunk.
Also, I thought the point of the wrapper script was to call the .py script using standard python?