topic Re: Lookups - using them to replace the host field... in Splunk Search

Lookups - using them to replace the host field...

Steve_Litras — Wed, 28 Apr 2010 00:54:35 GMT

Prior to 4.1, my host field reverse resolved (i.e. instead of ip addresses, it showed hostnames from DNS) for syslog data. This stopped when I upgraded to 4.1, so I'm trying to get it working again via lookups. I've got the lookup working when I direct the output to a new field:

dnslookup ip AS host OUTPUT host AS hostname

works fine, creates a new field "hostname" with the hostname data in it.

But when I just do:

dnslookup ip AS host OUTPUT host

The host field gets eliminated. What am I missing here? I really want my data to all have consistent host fields (i.e. only hostnames where they resolve, IP addresses if they can't).

Thanks Steve

Re: Lookups - using them to replace the host field...

gkanapathy — Wed, 28 Apr 2010 01:13:07 GMT

It's probably better to just set connection_host = dns to your UDP input stanza to resolve the old behavior. The lookup script you have is rather expensive to run on every unique IP on every single search.

You can not use a lookup to overwrite an incoming field value. If you really want to do this, set up a FIELDALIAS on the original field, and pass in the alias to the lookup. Again, not recommended for the long term.

Re: Lookups - using them to replace the host field...

Steve_Litras — Wed, 28 Apr 2010 08:44:22 GMT

Good call - I missed that in the doc.

Re: Lookups - using them to replace the host field...

Lowell — Fri, 30 Apr 2010 22:31:16 GMT

For anyone actually trying to replace host, as Steve was asking about.

Here is one way to workaround the issue without doing all the FIELDALIAS stuff:

... | eval ip=host | lookup dnslookup ip OUTPUT host AS hostname | eval host=coalesce(hostname,host) | eval ip=null()

If you have to use this in a number of your searches, you should consider making a macro for this

Note that if you already have the field ip, you can simply swap it out for a different (unused) field name. This approach should allow any failed lookups to preserve their original host value, which is what you want if your are dealing with a mix of hostnames and ip address in the host field.

If you have a small list of hosts that you are renaming (like in this example, where due to a setup issue Steve now has historical data with ip address instead of hostnames) then it probably makes more sense to make a new static lookup table (make a small .csv file in the lookups directory), rather than using an external lookup script, since using a static lookup file would be much faster.