Need assistance creating a file that can be updated with hosts to exclude from a search

[FPERVIL]

I have a query where I'm looking for users who are performing large file transfers (>50MB).  This query runs every day and as a result we have hosts that are legit.  These hosts names are extracted from the dst_host field of the results from my search.  As we compile a list of valid hosts, we can simply add that to the query to be excluded from the search like: 

index=* sourcetype=websense* AND (http_method="POST" OR http_method="PUT" OR http_method="CONNECT") AND bytes_out>50000000 NOT (dst_host IN (google.com, webex.com, *.zoom.us) OR dst_ip=1.2.3.4)

I know there's a better way to add the excluded host or IPs in a file that I can query against to exclude but I'm not sure how to do that.  I don't want to update the query everyday with hosts that should be excluded but rather a living document that can be updated with hosts or IPs that should excluded.

Can someone send point me in the right direction for this issue.

[scelikok · Accepted solution]

Hi @FPERVIL,

You can create two csv lookup files like below;

legit_hosts.csv (field name is dst_host)

dst_host
google.com
webex.com
*.zoom.us

legit_ips.csv (field name is dst_ip)

dst_ip
1.2.3.4

And use below query;

index=* sourcetype=websense* (http_method="POST" OR http_method="PUT" OR http_method="CONNECT") bytes_out>50000000 NOT [| inputlookup legit_hosts.csv ] NOT [|inputlookup legit_ips.csv ]

If you need to add/remove legit hostnames or IP addresses you can just update related lookup files.