Below is the props.conf at $SPLUNK_HOME/etc/system/default:
[SPLUNK_SERVICE_Log]
lookup_table = namelookup Id OUTPUT Name
I added same entry under $SPLUNK_HOME/etc/apps/search/default But it didnt made any difference and the error persist
Below is the transforms.conf at $SPLUNK_HOME/etc/system/default:
[namelookup]
external_cmd = namelookup.py Id Name
external_type = python
fields_list = Id, Name
However, when I run below search, I get "Error in 'lookup' command: The lookup table 'name_lookup' does not exist."
source="Test_Log.txt" | xmlkv entry | lookup namelookup Id OUTPUT Name | table Id, name
Script location :
$SPLUNK_HOME/etc/apps/search/lookups/namelookup.py
The script runs perfectly well as standalone program but gives error when called from Splunk. Any idea why this is happening? I restarted Splunk after I modified props.conf/transforms.conf. This is on version 4.1.5.
Thanks!
Did you try this search which leave off the output field:
source="Test_Log.txt" | xmlkv entry | lookup namelookup Id | table Id, name
I would title it troubleshooting lookup python script
wondering if their is a way to debug the lookup script with the help of breakpoints to figure out where exactly the script is failing. I would like debugger to trace it right from invocation of splunk search query. Not sure if its possible
Thanks Please suggest what should i title new post so that atleast someone provide solution
no not really but maybe do a new post for someone more familiar with python to look at it
you mean omit "OUTPUT Name" in the lookup command. I just tried that still search query doesnt return any results for name. Not sure where i am going wrong or missing something. Do you see any errors in the lookup script i.e. namelookup.py in the above section
yes as local is anything for customised configuration and will not be overwritten or deleted if you upgrade.And has precedence on other configurations.
Well $Splunk_HOME\etc\system\local doesnt have any existing props.conf or transforms.conf files. So are you suggesting to create new props.conf, transforms.conf there
Thanks the error is gone now but still it doesnt display the result. Please take a look at the script and let me know where its going wrong. Due to size restriction i cannot paste the script in comment box so i will paste it by "Answer Your Question" so that you can take a look at it
I would put .conf files in system/local rather than default and check permissions.
And the script in system/bin
Sorry to paste the script here, as their is a size restriction in comment box. Here is the script which doesnt return any search results. Please note i have hardcoded the values in Dictionary which should return values in search results. Please let me know where i am going wrong in the script
# File namelookup.py
# ------------------------------
import os,csv
#import pyodbc
import sys
def main():
if len(sys.argv) != 3:
print "Usage: python name_lookup.py [id field] [name field]"
sys.exit(0)
idf = sys.argv[1]
namef = sys.argv[2]
r = csv.reader(sys.stdin)
w = None
header = []
first = True
d1 = {}
# Add items
d1["006981166"] = "John"
d1["007094117"] = "Mike"
d1["007094118"] = "Scott"
for line in r:
if first:
header = line
print "Header:", header
if idf not in header or namef not in header:
print "Id and Name fields must exist in CSV data"
sys.exit(0)
csv.writer(sys.stdout).writerow(header)
w = csv.DictWriter(sys.stdout, header)
first = False
continue
# Read the result
result = {}
i = 0
while i < len(header):
if i < len(line):
result[header[i]] = line[i]
else:
result[header[i]] = ''
i += 1
# Perform the lookup
if len(result[idf]) and len(result[namef]) :
w.writerow(result)
elif len(result[idf]):
result[namef] = lookup(result[idf], d1)
if len(result[namef]):
w.writerow(result)
# Given a Id, find its Name
def lookup(id, d1):
try:
for key in d1.keys():
if key == id:
#print "Value=", d1[key]
return d1[key]
except:
return []
main()