Splunk Search

Error in 'lookup' command: The lookup table 'namelookup' does not exist.

Path Finder

Below is the props.conf at $SPLUNK_HOME/etc/system/default:

[SPLUNK_SERVICE_Log]
lookup_table = namelookup Id OUTPUT Name

I added same entry under $SPLUNK_HOME/etc/apps/search/default But it didnt made any difference and the error persist

Below is the transforms.conf at $SPLUNK_HOME/etc/system/default:

[namelookup]
external_cmd = namelookup.py Id Name
external_type = python
fields_list = Id, Name

However, when I run below search, I get "Error in 'lookup' command: The lookup table 'name_lookup' does not exist."

source="Test_Log.txt" | xmlkv entry | lookup namelookup  Id OUTPUT Name | table Id, name

Script location :

$SPLUNK_HOME/etc/apps/search/lookups/namelookup.py

The script runs perfectly well as standalone program but gives error when called from Splunk. Any idea why this is happening? I restarted Splunk after I modified props.conf/transforms.conf. This is on version 4.1.5.

Thanks!

Tags (2)

Motivator

Did you try this search which leave off the output field:

source="Test_Log.txt" | xmlkv entry | lookup namelookup Id | table Id, name

Motivator

I would title it troubleshooting lookup python script

0 Karma

Path Finder

wondering if their is a way to debug the lookup script with the help of breakpoints to figure out where exactly the script is failing. I would like debugger to trace it right from invocation of splunk search query. Not sure if its possible

Path Finder

Thanks Please suggest what should i title new post so that atleast someone provide solution

Motivator

no not really but maybe do a new post for someone more familiar with python to look at it

0 Karma

Path Finder

you mean omit "OUTPUT Name" in the lookup command. I just tried that still search query doesnt return any results for name. Not sure where i am going wrong or missing something. Do you see any errors in the lookup script i.e. namelookup.py in the above section

Motivator

yes as local is anything for customised configuration and will not be overwritten or deleted if you upgrade.And has precedence on other configurations.

0 Karma

Path Finder

Well $Splunk_HOME\etc\system\local doesnt have any existing props.conf or transforms.conf files. So are you suggesting to create new props.conf, transforms.conf there

Path Finder

Thanks the error is gone now but still it doesnt display the result. Please take a look at the script and let me know where its going wrong. Due to size restriction i cannot paste the script in comment box so i will paste it by "Answer Your Question" so that you can take a look at it

Motivator

I would put .conf files in system/local rather than default and check permissions.

And the script in system/bin

0 Karma

Path Finder

Sorry to paste the script here, as their is a size restriction in comment box. Here is the script which doesnt return any search results. Please note i have hardcoded the values in Dictionary which should return values in search results. Please let me know where i am going wrong in the script

    # File namelookup.py
# ------------------------------
import os,csv
#import pyodbc
import sys
def main():
      if len(sys.argv) != 3:
        print "Usage: python name_lookup.py [id field] [name field]"
        sys.exit(0)
      idf = sys.argv[1]
      namef = sys.argv[2]
      r = csv.reader(sys.stdin)
      w = None
      header = []
      first = True
      d1 = {}
      # Add items
      d1["006981166"] = "John"
      d1["007094117"] = "Mike"
      d1["007094118"] = "Scott"
      for line in r:
        if first:
            header = line
            print "Header:", header
            if idf not in header or namef not in header:
                print "Id and Name fields must exist in CSV data"
                sys.exit(0)
            csv.writer(sys.stdout).writerow(header)
            w = csv.DictWriter(sys.stdout, header)
            first = False
            continue

        # Read the result
        result = {}
        i = 0
        while i < len(header):
            if i < len(line):
                result[header[i]] = line[i]
            else:
                result[header[i]] = ''
            i += 1

        # Perform the lookup 
        if len(result[idf]) and len(result[namef]) :
            w.writerow(result)

        elif len(result[idf]):
        result[namef] = lookup(result[idf], d1)
            if len(result[namef]):
                w.writerow(result)




# Given a Id, find its Name
def lookup(id, d1):
     try:        
         for key in d1.keys():
        if key == id:
            #print "Value=", d1[key]
            return d1[key]       
     except:
        return []

main()