Scripted Input - Python Module Error

[nakiamatthews]

I have a python script that pulls data from an SFTP source and writes the data to a file (myScript.py). The script imports two modules that I installed. As such, I wrote a bash wrapper script to call my script and run it with standard python (myScriptLauncher.sh).

I can run the bash script from the command line fine. However, when I try to run it using Splunk (./splunk cmd /opt/splunk/etc/apps/myApp/bin/myScriptLauncher.sh), I get a module import error.

I'm not sure what I am doing wrong. Anyone have any suggestions?

error

Traceback (most recent call last):
  File "./myScript.py", line 4, in <module>
    import pysftp
ImportError: No module named pysftp

myScript.py

#!/usr/bin/env python

import json
import pysftp
import openpyxl

REST OF CODE GOES HERE

myScriptLauncher.sh

#!/bin/bash

echo myScript is starting >&2

cd $( dirname "${BASH_SOURCE[0]}" )

exec /usr/bin/python ./myScript.py

[badarsebard]

Creating a wrapper to call a Python installation on the host has been the conventional Splunk wisdom for years, as was bundling needed modules into the app. Both these approaches have numerous problems though. Using the wrapper is typically temperamental to permissions, environment variables, and in the case of the default system Python possibly versions of Python even older than 2.7.

My suggestion is to check out a new app I've written called PyDen which allows you to create Python virtual environments and install PyPI packages to the environment. You can then use these environments to execute custom commands and scripted inputs.

This provides three key advantages:

1. You can pick the version of Python you'd like to write in, including 3.5 to 3.7
2. Create an isolated environment that won't interfere with imports in other apps (this can happen when packages are bundled into apps)
3. Install any package available on PyPI to the environment

[kamlesh_vaghela]

@nakiamatthews

I think you have installedpysftp files in local python. So you should add path in your myScript.py script OR you have to copy those file into splunk lib.

See below ref link for your more info:

https://answers.splunk.com/answers/222934/modular-input-that-need-additional-python-modules.html

https://answers.splunk.com/answers/220196/import-non-native-python-libraries-into-splunk.html#answer...
https://answers.splunk.com/answers/8/can-i-add-python-modules-to-the-splunk-environment.html

[nakiamatthews]

I was under the impression that adding the modules to the splunk lib was not future proof - i.e. it will break whenever you upgrade Splunk.

[nakiamatthews]

Also, I thought the point of the wrapper script was to call the .py script using standard python?