Why does my external lookup script not work when called from the search head?

[fatemabwudel]

Hi,

So, I have set up an external lookup script, following the example of external_lookup.py that is shipped with Splunk by default.

My external lookup script, however, calls another perl script that does a postgres DB lookup of the fields passed to it.
The snippet of the external lookup script (mac_info.py written in python that calls "ip" which is written in perl) is:

#!/usr/bin/env python

import sys, os
import csv
import subprocess, datetime
import string

def hostlookup(ip):
    if ip is None:
        print >> sys.stderr, "ERROR No client_ip provided ..."
        return ''

    try:
        output = subprocess.check_output(['./ip',str(ip)])  # calls ip (perl script)
        #print "Inside try"
        #print output
        output= "".join(c for c in output if c not in ('"',','))

        return output

    except:
        print >> sys.stderr, "ERROR unknown exec: "
        return ''

The script works fine when run from the command line:

$ /opt/splunk/bin/python mac_info.py host ip < input.csv
host,ip
Mac address: xx.xx.xx.xx.xx.xx ,192.168.z.y
Mac address: yy.yy.yy.yy.yy.yy ,192.168.x.y

But when called from the search head, it does not show any results for "host" field, also no errors pop up.
Have read some posts in past saying that the Splunk python modules are limited and have to write some kind of wrappers to make it work. But don't really know what modules (perl/python) causing problems and how to get around to it.

P.S: I have put the scripts (both perl n python) in /opt/splunk/etc/system/bin/ dir and added a parameter in transforms.conf like:

[ip2mac]
external_cmd = mac_info.py host ip
fields_list = host, ip

Any help appreciated.

Thanks!

[ppeterson]

Something that's bitten me in the past, are you testing from the CLI as the same user that Splunk is running as?

[fatemabwudel]

Yep, Splunk runs as splunk user, and I am testing it from CLI a splunk user.
The problem is, I am not getting any errors while running from the search command on SH,
hence don't know what to troubleshoot.

[somesoni2]

Check if you've placed the script in correct location.
http://docs.splunk.com/Documentation/Splunk/6.5.2/Search/Writeasearchcommand#Search_command_executab...

[fatemabwudel]

Thanks for the suggestion.
I have enabled the dnslookup script that is shipped with Splunk by default, and resides in the same
location where I have put my custom external lookup script (/opt/splunk/etc/system/bin), and
it works just fine, i.e whenever I do a dnslookup using that external script, I always get fields populated.
Hence thought the custom script should also work in the same dir.

[somesoni2]

I would generally avoid updating anything in etc/system directory unless necessary. Custom stuffs should be in etc/apps in their own custom apps.

[fatemabwudel]

Just tested the custom scripts by putting them in /opt/splunk/etc/searchscripts/ dir, but no results.