Use lookup and stop matching for each entry in lookup

[gerald_contrera]

Hi All,

I am trying to use a lookup to check how many domains in a white list are actually being used.

The CSV has a single column with a list of domains

lookup CSV EG:
microsoftonline.com
gov.au
windows.net

I am running the following query:
index=gateway

| search dhost [| inputlookup DevGlobalWhitelist_lookup.csv | rename domain as dhost | fields + dhost ]
| dedup dhost | table dhost

This query somewhat works, but it doesn't stop when it matches in the lookup CSV
EG Results:
login.windows.net
adminwebservice.microsoftonline.com
login.microsoftonline.com
secure.aadcdn.microsoftonline-p.com

The problem is the gateway doesnt have a field for the top domain, so we are getting multiple *microsoftonline.com events for example.

dhost is the domain field from the gateway logs.

Ideally i would like the query to dedup the field in the CSV lookup, so that once any results matches to the CSV file it will not check against that again.

What am i doing wrong?

Thank you in advance

[jkat54]

Can you try this?

index=gateway  [| inputlookup DevGlobalWhitelist_lookup.csv | rename domain as dhost | eval dhost='*'.dhost.'*' | fields dhost ]
| dedup dhost 
| table dhost