Hi All,
I am trying to use a lookup to check how many domains in a white list are actually being used.
The CSV has a single column with a list of domains
lookup CSV EG:
microsoftonline.com
gov.au
windows.net
I am running the following query:
index=gateway
| search dhost [| inputlookup DevGlobalWhitelist_lookup.csv | rename domain as dhost | fields + dhost ]
| dedup dhost | table dhost
This query somewhat works, but it doesn't stop when it matches in the lookup CSV
EG Results:
login.windows.net
adminwebservice.microsoftonline.com
login.microsoftonline.com
secure.aadcdn.microsoftonline-p.com
The problem is the gateway doesnt have a field for the top domain, so we are getting multiple *microsoftonline.com events for example.
dhost is the domain field from the gateway logs.
Ideally i would like the query to dedup the field in the CSV lookup, so that once any results matches to the CSV file it will not check against that again.
What am i doing wrong?
Thank you in advance
Can you try this?
index=gateway [| inputlookup DevGlobalWhitelist_lookup.csv | rename domain as dhost | eval dhost='*'.dhost.'*' | fields dhost ]
| dedup dhost
| table dhost