topic Re: Python script. Module not loading when executed by splunk in Splunk Dev

Python script. Module not loading when executed by splunk

ryan461 — Mon, 01 Oct 2012 15:03:13 GMT

I have a python script on our splunk server to ingest data from a database using pymssql and _mssql. I added this script through the web gui in Manager > Data Inputs > Scripts. I can run the script manually just fine. But when its executed by splunk, in the splunkd logs it says ImportError: No module named _mssql.

The pymssql/_mssql package was installed manually.

Re: Python script. Module not loading when executed by splunk

alacercogitatus — Mon, 01 Oct 2012 15:33:46 GMT

Splunk uses it's own python to execute scripts. So if Splunk Python doesn't know about _mssql, it won't find it. Did you install _mssql to the file system? Try running manually with splunk cmd python /path/to/your/script. I got around that by using a sh script to call the python script with #!/usr/bin/python as the executable to use.

Re: Python script. Module not loading when executed by splunk

ryan461 — Mon, 01 Oct 2012 15:48:45 GMT

That does get it to execute, the data being ingested was being printed, which splunk would read. But now that its a shell script executing it, i imagine ill have to write to a file or something?

Re: Python script. Module not loading when executed by splunk

alacercogitatus — Mon, 01 Oct 2012 15:51:29 GMT

You could, but I just use the python print command, and splunk eats it just fine. I used the exec command from the sh script and it works great.

Re: Python script. Module not loading when executed by splunk

dalglish — Tue, 02 Oct 2012 06:20:22 GMT

I just use wrapper python script with non-splunk python with installed modules in scripted input. (taken from here) Something like that:

import os, sys
for envvar in ("PYTHONPATH", "LD_LIBRARY_PATH"):
    if envvar in os.environ:
        del os.environ[envvar]
python_executable = "D:\Python27\python.exe"
real_script = "path_to_script"
os.execv(python_executable, [ python_executable, real_script ] + sys.argv[1:])

Re: Python script. Module not loading when executed by splunk

asifhj — Fri, 27 Sep 2013 14:18:34 GMT

Guys,

Instead setting all those parameters and creating subproccess.
Simply copy your required packages/files from

/usr/local/lib/python2.7/dist-packages

to

/opt/splunk/lib/python2.7/site-packages/

Re: Python script. Module not loading when executed by splunk

alacercogitatus — Fri, 27 Sep 2013 14:26:25 GMT

Possible, but if you upgrade, you may lose the files.

Re: Python script. Module not loading when executed by splunk

David — Tue, 29 Nov 2016 17:37:06 GMT

I downvoted this post because this is not supported, and as a comment suggested, will break between upgrades (and also not be transferrable between systems)