topic Issue with Python modules when attempting to execute app script. "ImportError: No module named searchcommands" in Splunk Dev

Issue with Python modules when attempting to execute app script. "ImportError: No module named searchcommands"

evan_roggenkamp — Wed, 21 Sep 2016 17:45:33 GMT

I am trying to use an app that appears to call some Python scripts to get the data into Splunk.

Splunk is having some trouble with the Python modules required to run the script, and I am not quite sure how to solve it using Splunk's own Python environment. (I could probably fix this if Splunk used a standalone Python environment, but I do not know how to get it to do that)

Here is the log from the job inspector:

09-21-2016 12:09:48.162 INFO  ChunkedExternProcessor - Running process: "C:\Program Files\Splunk\bin\python.exe" "C:\Program Files\Splunk\etc\apps\TA-prtg\bin\prtglivedata.py"
09-21-2016 12:09:48.340 ERROR ChunkedExternProcessor - EOF while attempting to read transport header
09-21-2016 12:09:48.342 ERROR ChunkedExternProcessor - Error in 'prtglivedata' command: External search command exited unexpectedly with non-zero error code 1.
09-21-2016 12:09:48.342 INFO  UserManager - Unwound user context: admin -> NULL
09-21-2016 12:09:48.345 ERROR ChunkedExternProcessor - stderr: Traceback (most recent call last):
09-21-2016 12:09:48.346 ERROR ChunkedExternProcessor - stderr:   File "C:\Program Files\Splunk\etc\apps\TA-prtg\bin\prtglivedata.py", line 32, in <module>
09-21-2016 12:09:48.346 ERROR ChunkedExternProcessor - stderr:     from splunklib.searchcommands import dispatch, GeneratingCommand, Configuration, Option, validators
09-21-2016 12:09:48.346 ERROR ChunkedExternProcessor - stderr: ImportError: No module named searchcommands

I tried to run the Python script directly using Splunk's Python, but that throws an error as well:

c:\Program Files\Splunk\bin>splunk cmd python "C:\Program Files\Splunk\etc\apps\TA-prtg\bin\prtglivedata.py"
Traceback (most recent call last):
  File "C:\Program Files\Splunk\etc\apps\TA-prtg\bin\prtglivedata.py", line 32, in <module>
    from splunklib.searchcommands import dispatch, GeneratingCommand, Configuration, Option, validators
ImportError: No module named searchcommands

Re: Issue with Python modules when attempting to execute app script. "ImportError: No module named searchcommands"

richard_wilhite — Thu, 22 Sep 2016 15:46:47 GMT

We run standard python by calling python and the script we want in a bash script. Such as:

#!/bin/bash
/usr/bin/python /opt/splunk/bin/scripts/foo.py

Though, this would be a .bat script for you, since looks like you are on Windows. You would also need to ensure the Windows server had a python install on it. By using the standard python install on your server, you can then add modules just like any other python system.

Also, be aware, that running the script this way will require any paths you have in your py script to be absolute and not a relative reference.

Re: Issue with Python modules when attempting to execute app script. "ImportError: No module named searchcommands"

evan_roggenkamp — Thu, 22 Sep 2016 17:37:55 GMT

The app is somehow calling the python script in question. I am not sure how I would tell this app to use a .bat file in place of trying to run:

 09-21-2016 12:09:48.162 INFO  ChunkedExternProcessor - Running process: "C:\Program Files\Splunk\bin\python.exe" "C:\Program Files\Splunk\etc\apps\TA-prtg\bin\prtglivedata.py"

Is there no other way to tell splunk to use regular non-splunk python?

The other idea I had was creating some kind of wrapper where I copy and rename prtglivedata.py, and then use one copy to call the other with the correct environment, but this seems messy as well.

Re: Issue with Python modules when attempting to execute app script. "ImportError: No module named searchcommands"

richard_wilhite — Fri, 23 Sep 2016 20:07:57 GMT

Someone else may be better suited to answer this question than I. However, I hate to leave you hanging. Looking at your output, above, more closely, I see the ImportError is for searchcommands. Which is part of the Splunk SDK, I believe. I would assume the necessary SDK modules would be built into the app. But maybe that is the next thing to look at.

Re: Issue with Python modules when attempting to execute app script. "ImportError: No module named searchcommands"

ddrillic — Sat, 24 Sep 2016 23:56:49 GMT

A related thread about the error - ERROR ChunkedExternProcessor - stderr: ImportError: No module named searchcommands

At Why am I getting error "No module named splunklib.searchcommands" trying to run a custom search command?

Re: Issue with Python modules when attempting to execute app script. "ImportError: No module named searchcommands"

adambloom — Tue, 29 Sep 2020 11:06:02 GMT

You'll need to place the splunk-sdk python library in {{SPLUNK_HOME}}/lib/python-2.7/site-packages or locally in {{SPLUNK_HOME}}/etc/apps/TA-prtg/bin. Download it from here: http://dev.splunk.com/python. You should then be able to import packages from splunklib in the default Splunk python interpreter.

Re: Issue with Python modules when attempting to execute app script. "ImportError: No module named searchcommands"

evan_roggenkamp — Mon, 26 Sep 2016 13:24:13 GMT

I think we are on the right track with that thought.
Thanks Richard 🙂

Re: Issue with Python modules when attempting to execute app script. "ImportError: No module named searchcommands"

adambloom — Mon, 26 Sep 2016 15:10:20 GMT

evan - see my comment above. Somehow ended up above this and not below.