external_lookup example: Could not find all of the specified lookup fields in the lookup table

[are0002]

I got this error when I configure an automathic lookup:
Could not find all of the specified lookup fields in the lookup table

I tried the script and runs perfectly
I created a partial CSV file and execute it with the python srcript:
/opt/splunk/bin/splunk cmd python /opt/splunk/etc/system/bin/external_lookup.py host ip < temp.csv
host,ip
www.hardware.fr,83.243.20.80
www.bash.org,69.61.106.93
www.somafm.com,64.147.167.20

Then I configured the trasnforms.conf:

and the props.conf:
[mydnslookup]
external_cmd = external_lookup.py host ip
external_type = python
fields_list = dst_ip, dst_hostname

[pan_traffic]
LOOKUP-rdns = mydnslookup ip AS dst_ip OUTPUT host AS resolved_host

also I tried:

[pan_traffic]
LOOKUP-rdns = mydnslookup clientip AS dst_ip OUTPUT host AS resolved_host

I used this tutorial:
http://docs.splunk.com/Documentation/Splunk/latest/knowledge/Addfieldsfromexternaldatasources#Set_up...

[are0002]

Hi Iguinn,

You were right. The field_list refers to the CSV headers instead of splunk data fields. Thank you for your answer.

[lguinn2 · Accepted solution]

Try

fields_list = host, ip

instead of

fields_list = dst_ip, dst_hostname

And

[pan_traffic]
LOOKUP-rdns = mydnslookup ip AS dst_ip OUTPUT host AS resolved_host

is correct. I assume that the [pan_traffic] stanza is in props.conf and the [mydnslookup] is in transforms.conf

Finally, I assume that the dst_ip field exists in your pan_traffic sourcetype. It can't be a field that you have extracted with rex. AND, order is important: field definition for dst_ip should precede the lookup in props.conf

[are0002]

I am trying to do a reverse dns resolution (obtain the domain name with the IP address).
I am using the external_lookup.py script that is able to do dns resolutions in both ways. I need to resolve the dns dynamicaly that's why I need an external script.

[lguinn2]

First, I don't think I understand what you are trying to do.

If you want to have Splunk look up IP addresses in a fixed file, you don't need external_lookup.py

A file-based lookup is much easier than what you are doing. There is documentation for creating a file-based lookup at Setup a fields lookup based on a static file . The documentation shows how to edit props.conf and transforms.conf

But you can do this very easily from the Manager UI:

1. Build a CSV file on your desktop that contains the data you need. The first row MUST be a header; the column names will be the field names in your lookup.
2. Go to Manager » Lookups in the Splunk UI
3. Add a new Lookup Table File. This is where you will upload the CSV file from your desktop into Splunk.
4. Add a new Lookup Definition. This is where you tell Splunk that you want to do a file-based lookup, using your Lookup Table file from the previous step.
5. Add a new Automatic Lookup. Here you tell Splunk how to use your Lookup Definition automatically, and tell it which fields to retrieve, etc.
6. Be sure to set permissions on each of the items: the table, the definition, and the automatic lookup - if you want them to be used by others.

Once you create the lookup in the UI, you can see what it does in props.conf and transforms.conf - but no need to create them yourself.