Getting Data In

Why does my external lookup script not work when called from the search head?

fatemabwudel
Path Finder

Hi,

So, I have set up an external lookup script, following the example of external_lookup.py that is shipped with Splunk by default.

My external lookup script, however, calls another perl script that does a postgres DB lookup of the fields passed to it.
The snippet of the external lookup script (mac_info.py written in python that calls "ip" which is written in perl) is:

#!/usr/bin/env python

import sys, os
import csv
import subprocess, datetime
import string

def hostlookup(ip):
    if ip is None:
        print >> sys.stderr, "ERROR No client_ip provided ..."
        return ''

    try:
        output = subprocess.check_output(['./ip',str(ip)])  # calls ip (perl script)
        #print "Inside try"
        #print output
        output= "".join(c for c in output if c not in ('"',','))

        return output

    except:
        print >> sys.stderr, "ERROR unknown exec: "
        return ''

The script works fine when run from the command line:

$ /opt/splunk/bin/python mac_info.py host ip < input.csv
host,ip
Mac address: xx.xx.xx.xx.xx.xx ,192.168.z.y
Mac address: yy.yy.yy.yy.yy.yy ,192.168.x.y

But when called from the search head, it does not show any results for "host" field, also no errors pop up.
Have read some posts in past saying that the Splunk python modules are limited and have to write some kind of wrappers to make it work. But don't really know what modules (perl/python) causing problems and how to get around to it.

P.S: I have put the scripts (both perl n python) in /opt/splunk/etc/system/bin/ dir and added a parameter in transforms.conf like:

[ip2mac]
external_cmd = mac_info.py host ip
fields_list = host, ip

Any help appreciated.

Thanks!

0 Karma

ppeterson
Path Finder

Something that's bitten me in the past, are you testing from the CLI as the same user that Splunk is running as?

0 Karma

fatemabwudel
Path Finder

Yep, Splunk runs as splunk user, and I am testing it from CLI a splunk user.
The problem is, I am not getting any errors while running from the search command on SH,
hence don't know what to troubleshoot.

0 Karma

somesoni2
SplunkTrust
SplunkTrust
0 Karma

fatemabwudel
Path Finder

Thanks for the suggestion.
I have enabled the dnslookup script that is shipped with Splunk by default, and resides in the same
location where I have put my custom external lookup script (/opt/splunk/etc/system/bin), and
it works just fine, i.e whenever I do a dnslookup using that external script, I always get fields populated.
Hence thought the custom script should also work in the same dir.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

I would generally avoid updating anything in etc/system directory unless necessary. Custom stuffs should be in etc/apps in their own custom apps.

0 Karma

fatemabwudel
Path Finder

Just tested the custom scripts by putting them in /opt/splunk/etc/searchscripts/ dir, but no results.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...