Splunk Search

Search with lookup and several fields to match

mcayrol
Explorer

Hi splunkers,

After several days to be block with an issue regarding lookup, I try to have a little help here,

Here is my problem, I have an asset which brings me alerts, sometime the same alert so I want to exclude the duplicates ones, for this, I create a lookup that save the research of my alert's query. This part work great, and after I want to exclude the event that is not matching ALL the field of the lookup.

So my lookup is like this : duplicate.csv

subject,source,dest,malware
null,null,null,null

The field of my search event has the SAME name that my lookup field.

Here is my query at the moment

index=xxxxxx
| search NOT [| lookup duplicate.csv subject AS source,dest AS dest,malware AS malware

| outputlookup append=true duplicate.csv


I don't know how to create the link between search field and lookup field because they share the same name.

And I don't now how I do to display the event ONLY if they match all the field in my lookup (4)


Thank for your help. This is a lovely community 🙂

 

 

Labels (2)

renjith_nair
SplunkTrust
SplunkTrust

Try below and check if you get expected result. Once confirmed, you may add outputlookup

 

index=xxxxxx NOT [| inputlookup duplicate.csv]

 

0 Karma

mcayrol
Explorer

I've try but I got the same result as before

The search found me all the event and not exclude one

when I do index=xxxxxx NOT  lookup duplicate.csv

I got 0 event, all are exclude

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

Just to be sure , its inputlookup and not lookup -

can you try this and look at the job inspector for the search log. You should be able to see the final search it executes

index=xxxxxx NOT [| inputlookup duplicate.csv]

 

It should be converted to a search similar to

index=xxxxxx (NOT subject="subject value" OR source = "source value" OR dest ="dest value" OR malware="malware value") (NOT subject=".....)

and repeat for all the values inside the lookupfile

A run anywhere example would be

|makeresults|eval country="Aruba Angola China Japan"|makemv country|mvexpand country
|search NOT [|inputlookup geo_attr_countries where (country = "Aruba" OR country="Angola") |fields country]

 

mcayrol
Explorer

Ohhh indeed it's Inputlookup, I just saw the differents in the documentation,

I run the search but since I have a lookup with 500 rows, the search in Job inspector is very huge and like you say, I see (NOT field3lookup ="randomrowfiled3lookup" OR NOT ...)

But despite this, nothing changes compare at the result of a basic "index=XXXX" search

I got the same numbers of rows.

 

 

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.