topic Re: How do I add a python module for external lookup command? in Splunk Dev

How do I add a python module for external lookup command?

ndoshi — Mon, 27 Jun 2011 03:25:27 GMT

I know the answers site has a description for having one Python program call another, which includes a custom module, but this seems to be more geared for scripted inputs or alerts. http://splunk-base.splunk.com/answers/8/can-i-add-python-modules-to-the-splunk-environment. I have a custom module that works outside of Splunk in Python 2.6, but I need to use this within a lookup Python script. The script will have parameters passed within it as labeled in transforms.conf. The custom module will eventually call native C/C++ code. The module builds itself into an egg file. I can't simply drop this file into Splunk's Python 2.6 site-packages directory as even the import didn't work. How do I let Splunk know about this module so that the lookup Python script can import and use it?

Re: How do I add a python module for external lookup command?

BryantD — Tue, 28 Jun 2011 19:40:53 GMT

Try using a wrapper script. The one described in the answer you link to is possibly more complex than what's needed for lookups. I solved this problem yesterday using a wrapper, as follows.

wrapper.py:

import os

NEW_PYTHON_PATH = '/usr/bin/python'

os.environ['PYTHONPATH'] = NEW_PYTHON_PATH
my_process = '/opt/splunk/etc/apps/zgp/bin/lookup.py')

os.execv(my_process, ())

Re: How do I add a python module for external lookup command?

ndoshi — Wed, 29 Jun 2011 00:32:23 GMT

Thanks for your reply. I think the the field header arguments would have to be put in the myprocess = ... part. Lowell provided an even easier answer in a past post for my module that happens to be in an egg: Lowell wrote "Instead I recommend creating .egg files, putting them in your own lib folder, and explicitly doing a sys.path.append("/your/lib/whatever.egg") before your import."

Re: How do I add a python module for external lookup command?

rosslord — Fri, 03 Feb 2012 21:18:48 GMT

As an aside, I was able to call a script with this:

#!/bin/bash
/usr/bin/python /path/to/python/script.py

.. andit found all my moduldes and worked fine.

Re: How do I add a python module for external lookup command?

ndoshi — Mon, 06 Feb 2012 14:12:15 GMT

My original question was for lookup scripts as lookups call Python directly and do not call shell scripts.