topic Re: having some trouble with a lookup table. in Getting Data In

having some trouble with a lookup table.

mloven — Thu, 29 Sep 2011 05:44:33 GMT

Hi all. I'm having some issues getting a lookup table to work properly. Here are the pertinent details:

I have a csv file (called combined.csv) in /opt/splunk/etc/apps/search/lookups that has two columns. It looks like this:

ipaddress,resolved
192.168.1.1,host1
192.168.1.2,host2
192.168.1.3,host3
and so on...

I have this in my /opt/splunk/apps/search/local/transforms.conf:

[dns_lookup]
filename = combined.csv
max_matches = 1

And this in my /opt/splunk/apps/search/local/props.conf:

[dns_lookup]
LOOKUP-dns = dns_lookup ipaddress OUTPUT resolved

I've restarted splunk multiple times.

Basically, I followed the instructions for using a static lookup table here but I'm not getting the results I was shooting for. I was hoping that the "resolved" field would show up when I did a search, but I'm seeing no new fields.

I don't really even know where to begin troubleshooting this one. Anyone have any suggestions?

Re: having some trouble with a lookup table.

hjwang — Mon, 28 Sep 2020 09:55:50 GMT

make sure you have the sourcetype dns_lookup, because [stanza_name] is the sourcetype, host, or source to which this lookup applies and can't use regex-type syntax.

Re: having some trouble with a lookup table.

gkanapathy — Thu, 29 Sep 2011 13:19:11 GMT

Well, you can actually use a regex-like syntax for [host::] and [source::] (and also sourcetype stanzas if you really must).

Re: having some trouble with a lookup table.

mloven — Thu, 29 Sep 2011 20:25:16 GMT

ok, so I changed my props.conf to

[syslog]
LOOKUP-dns = dns_lookup ipaddress OUTPUT resolved

I missed the part where that stanza name had to refer to the sourcetype (or source, host, etc). All of the events on this system are of the syslog sourcetype.

I've restarted again, but I'm still getting the same results.

EDIT... sorry... can't change the formatting in my comment. That props.conf stanza should look exactly like my original one, except the [dns_lookup] has been replaced by [syslog].

Re: having some trouble with a lookup table.

tgow — Thu, 29 Sep 2011 20:52:02 GMT

Make sure that field "ipaddress" is also in your data with a sourcetype of "syslog". The "ipaddress" field with a sourcetype of syslog has to match the first column in your combined.csv file of "ipaddress". Here is an example of what should work:

Sample syslog:
Sep 29 15:57:36 acmepayroll sshd[14867]: Failed password for invalid user rpm from 110.172.158.2 port 33907 ssh2

The "110.172.158.2" must be assigned to the field "ipaddress". If it is not then you can pull the field in the transforms.conf file.

props.conf:

    [syslog]
    LOOKUP-dns = dns_lookup ipaddress OUTPUTNEW resolved
    REPORT-dns = getip

transforms.conf

[dns_lookup]
filename = combined.csv
max_matches = 1
min_matches = 1

[getip]
REGEX = from\s+([^\s]+)
FORMAT = ipaddress::$1

Log back into Splunk and you should be able to run the following search:

sourcetype=syslog | table ipaddress, resolved

Re: having some trouble with a lookup table.

mloven — Thu, 29 Sep 2011 21:43:31 GMT

That was the ticket. I wasn't aware that the column name in my lookup table needed to be a valid field. I changed it to "host" and now it works like a champ.

Thanks all!