What's the best way to only do a Lookup based on the results of the main search?

[finchy]

Hi,

What's the best way to only do a Lookup based on the results of the main search?  I want to only run this when 2 fields don't match.  Pseudo would be

If field1!=field2 THEN | lookup accounts department as field2 OUTPUT

So like an if then statement most programming languages allow

Thanks

Lee

[bowesmana]

You CAN actually do conditional lookup, as long as your lookup is a CSV

https://docs.splunk.com/Documentation/Splunk/9.1.1/SearchReference/ConditionalFunctions#lookup.28.26...

I don't think it's very commonly used, but works well

... search...
| eval output=if(field1!=field2, 
          lookup("mylookup.csv", json_object("department", field2), 
                 json_array("output_field1","output_field2")), 
          "{}")

 You will get back a field output with a JSON representation of the output fields listed in the JSON array

[yuanliu]

Nice ref.  Thanks, @bowesmana! (Looks like something since 8.)

[fredclown]

You cannot do a conditional lookup, but you could do the lookup across all the data and then only conditionally display the data that was looked up.

[finchy]

Hi, i wanted to avoid doing a lookup if certain conditions are in place if that's not possible will just have to do it which returns the data if it finds any was trying to just save some cpu and time

[yuanliu · Accepted solution]

How to save CPU will depends on the actual flow of your search.  For example, if field1!=<fixed string pattern> is exceedingly rare in a large dataset, you can include the lookup in an append subsearch, like

<your main search>
| append
  [search <somesearch> field1 != <fixed string pattern>
   | lookup accounts department as field2 OUTPUT]

Then  manipulate the combined stream to utilize lookup output.

Because field1 != field2 is inapplicable in search command, this technique will not save you index search time.  However, if you have a situation where index search is cheap but lookup is exceedingly expensive (it can happen), you can still do it, like

<your main search>
| append
  [search <same main search>
   | where field1 != field2
   | lookup accounts department as field2 OUTPUT]

Alternatively, you are ONLY interested in annotating field2 for which field1 != field2, you can use appendpipe (which is very efficient)

<your main search>
| appendpipe
  [stats count by field1 field2
   | where field1 != field2
   | lookup accounts department as field2 OUTPUT
   | fields - count]

In all cases, you will need to massage these annotations back into your final results.  Hope this helps.