topic Re: Lookup OUTPUTNEW Behavior in Splunk Search

Lookup OUTPUTNEW Behavior

olivier_ma — Tue, 29 Sep 2020 19:57:50 GMT

Hello,

I have a wrong behavior on lookup command with OUTPUTNEW options.

What I'm trying to achieve is to update the host_name of an asset from source2, if source1 doesn't provide a host_name.

| lookup source1 host_ip OUTPUT host_name
| eval host_name= if(host_name=="NONE", NULL, host_name)
| lookup source2 host_ip OUTPUTNEW host_name

So for a specific IP, source1 provides a host_name, but I have "NONE" because source2 lookup doesn't know the host_name.

This behavior seems to be incorrect if we read the Splunk documentation :

If the OUTPUTNEW clause is specified, the lookup is not performed for events in which the output fields already exist.

So source2 lookup should not be performed.

Anyone has an idea about what's going on ?

Re: Lookup OUTPUTNEW Behavior

HiroshiSatoh — Mon, 11 Jun 2018 10:42:23 GMT

I think that EVAL is the cause.

↓"host_name" is added only to matching data.
 | lookup source1 host_ip OUTPUT host_name

↓"host_name" will be added to everything
 | eval host_name= if(host_name=="NONE", NULL, host_name)

↓Do not update because there is "host_name" in everything
| lookup source2 host_ip OUTPUTNEW host_name

Re: Lookup OUTPUTNEW Behavior

olivier_ma — Tue, 29 Sep 2020 19:57:59 GMT

Hi,

I'm not totally agree with you :
↓"host_name" is added if IP is known from source1
| lookup source1 host_ip OUTPUT host_name

↓if host_name is known from source1, then keep the host_name, otherwise, set it to NULL (which I hope that it will 'delete' the field)
| eval host_name= if(host_name=="NONE", NULL, host_name)

↓if host_name already exists, then do nothing, otherwise try to find a match
| lookup source2 host_ip OUTPUTNEW host_name

This behavior it's confirmed if I use only:
| lookup source1 host_ip OUTPUT host_name
| eval host_name= if(host_name=="NONE", NULL, host_name)
I have a host_name

But when I add
| lookup source2 host_ip OUTPUTNEW host_name
I have 'NONE' as host_name which is the behavior of no match (but lookup is performed despite of 'OUTPUTNEW' clause)

Re: Lookup OUTPUTNEW Behavior

jkat54 — Mon, 11 Jun 2018 11:28:13 GMT

Source 2 lookup will not “stomp” on events that already have host_name. The lookup will still be executed however.

So you should get the same results if you do the search without the eval.

Re: Lookup OUTPUTNEW Behavior

HiroshiSatoh — Tue, 29 Sep 2020 19:53:03 GMT

When is host_name added?
It does not delete the field. Just set the field to NULL.

＞↓if host_name is known from source1, then keep the host_name, otherwise, ＞set it to NULL (which I hope that it will 'delete' the field)
＞| eval host_name= if(host_name=="NONE", NULL, host_name)

So I think that this is OK.
| lookup source1 host_ip OUTPUT host_name
| lookup source2 host_ip OUTPUTNEW host_name