Splunk Search

Lookup command to both filter and append new fields

matthewb4
Path Finder

How do I use lookup command to filter events based on one of the fields but then just add the rest of the fields to the remaining events? For example, I want the events in my base search to be filtered by values that match field1 in the lookup file. But then I just want the field2 and field3 values from the lookup file to be added to the remaining events since these two fields don't exist in the base events.

how would i modify the below query so that it's not filtering by field2 and field3 as well but simply appending these values to the remaining events?

base search ... [|inputlookup partner.csv | fields field1 field2 field3 ]

0 Karma

starcher
SplunkTrust
SplunkTrust

That is a simple application of the lookup.

base search | lookup myLookup field1 OUTPUTNEW

elliotproebstel
Champion

Here are two ways to do this. Try both and see which is faster for your data sets:

base search [ | inputlookup partner.csv | fields field1 ] | lookup partner.csv field1 OUTPUT field2 field3

OR

base search | lookup partner.csv field1 OUTPUT field2 field3 | where isnotnull(field2) OR isnotnull(field3)

I'd expect the first option to work well if the size of your CSV is quite small compared to the number of events being searched in the base search. If the CSV is quite large and the base search alone does not return that many events, then the second might be faster.

starcher
SplunkTrust
SplunkTrust

I agree with the use of isnotnull if you want only events that had the returned values from the lookup.

0 Karma

starcher
SplunkTrust
SplunkTrust

But by what field are you wanting to lookup the remaining fields?

0 Karma

matthewb4
Path Finder

field1? if field1 value matches a value in the base search events, then I would want field2 and field3 from the corresponding row in lookup table to be added to this event

0 Karma
Get Updates on the Splunk Community!

Observability | How to Think About Instrumentation Overhead (White Paper)

Novice observability practitioners are often overly obsessed with performance. They might approach ...

Cloud Platform | Get Resiliency in the Cloud Event (Register Now!)

IDC Report: Enterprises Gain Higher Efficiency and Resiliency With Migration to Cloud  Today many enterprises ...

The Great Resilience Quest: 10th Leaderboard Update

The tenth leaderboard update (11.23-12.05) for The Great Resilience Quest is out >> As our brave ...