Getting Data In

Why does my external lookup script not work when called from the search head?

fatemabwudel
Path Finder

Hi,

So, I have set up an external lookup script, following the example of external_lookup.py that is shipped with Splunk by default.

My external lookup script, however, calls another perl script that does a postgres DB lookup of the fields passed to it.
The snippet of the external lookup script (mac_info.py written in python that calls "ip" which is written in perl) is:

#!/usr/bin/env python

import sys, os
import csv
import subprocess, datetime
import string

def hostlookup(ip):
    if ip is None:
        print >> sys.stderr, "ERROR No client_ip provided ..."
        return ''

    try:
        output = subprocess.check_output(['./ip',str(ip)])  # calls ip (perl script)
        #print "Inside try"
        #print output
        output= "".join(c for c in output if c not in ('"',','))

        return output

    except:
        print >> sys.stderr, "ERROR unknown exec: "
        return ''

The script works fine when run from the command line:

$ /opt/splunk/bin/python mac_info.py host ip < input.csv
host,ip
Mac address: xx.xx.xx.xx.xx.xx ,192.168.z.y
Mac address: yy.yy.yy.yy.yy.yy ,192.168.x.y

But when called from the search head, it does not show any results for "host" field, also no errors pop up.
Have read some posts in past saying that the Splunk python modules are limited and have to write some kind of wrappers to make it work. But don't really know what modules (perl/python) causing problems and how to get around to it.

P.S: I have put the scripts (both perl n python) in /opt/splunk/etc/system/bin/ dir and added a parameter in transforms.conf like:

[ip2mac]
external_cmd = mac_info.py host ip
fields_list = host, ip

Any help appreciated.

Thanks!

0 Karma

ppeterson
Path Finder

Something that's bitten me in the past, are you testing from the CLI as the same user that Splunk is running as?

0 Karma

fatemabwudel
Path Finder

Yep, Splunk runs as splunk user, and I am testing it from CLI a splunk user.
The problem is, I am not getting any errors while running from the search command on SH,
hence don't know what to troubleshoot.

0 Karma

somesoni2
Revered Legend
0 Karma

fatemabwudel
Path Finder

Thanks for the suggestion.
I have enabled the dnslookup script that is shipped with Splunk by default, and resides in the same
location where I have put my custom external lookup script (/opt/splunk/etc/system/bin), and
it works just fine, i.e whenever I do a dnslookup using that external script, I always get fields populated.
Hence thought the custom script should also work in the same dir.

0 Karma

somesoni2
Revered Legend

I would generally avoid updating anything in etc/system directory unless necessary. Custom stuffs should be in etc/apps in their own custom apps.

0 Karma

fatemabwudel
Path Finder

Just tested the custom scripts by putting them in /opt/splunk/etc/searchscripts/ dir, but no results.

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...