topic Re: Is it possible if a non matching domain is emailing us, it should display in a dashboard? in Splunk Search

Is it possible if a non matching domain is emailing us, it should display in a dashboard?

tonyxavierj — Wed, 09 Feb 2022 19:07:58 GMT

I am trying to explore more ways to check if business email compromise is being happening in our organization, just before the end user recognises it.

i have a list of domains that we usually communicate with, there are around 490 domains I have listed and added to a csv file. there is an index which is updated in realtime which have logs from mimecast. I would like to list out domains which are trying to establish email communication with our organization which are not there in the csv file.

so if a non matching domain is emailing us, it should display in a dashboard. is this possible?

Re: new email communication

richgalloway — Wed, 09 Feb 2022 15:05:29 GMT

A search for data in an index that is not in a CSV file would look something like this:

index=mimecast NOT [ | inputlookup domains.csv | return 1000 domain ]

I'm assuming the index and the CSV file use the same field names. If that is not the case then a rename will be needed after the inputlookup.

Re: new email communication

tonyxavierj — Thu, 10 Feb 2022 06:31:03 GMT

i tried with the following search

index= mimecast NOT [ | inputlookup Sender1.csv | return 1000 Sender ]
Sender1 have a list of email address and the field name is Sender

the results are same if i search with or without the filter. its is not omitting or removing the email address which are there in the csv file.

Re: new email communication

ITWhisperer — Thu, 10 Feb 2022 06:58:50 GMT

Does this work better?

index= mimecast NOT [ | inputlookup Sender1.csv | return 1000 Sender | format ]

Re: Is it possible if a non matching domain is emailing us, it should display in a dashboard?

tonyxavierj — Thu, 10 Feb 2022 07:11:37 GMT

gives error
Regex: regular expression is too large

if format is removed it gives results, but the results are same as without inputlookup

Re: Is it possible if a non matching domain is emailing us, it should display in a dashboard?

ITWhisperer — Thu, 10 Feb 2022 07:15:21 GMT

How many rows does the csv file have?

Re: Is it possible if a non matching domain is emailing us, it should display in a dashboard?

tonyxavierj — Thu, 10 Feb 2022 07:46:31 GMT

50000

Re: Is it possible if a non matching domain is emailing us, it should display in a dashboard?

ITWhisperer — Thu, 10 Feb 2022 07:57:50 GMT

Subsearches are limited to 50000 events - can you break the csv down into smaller files?

Re: Is it possible if a non matching domain is emailing us, it should display in a dashboard?

tonyxavierj — Thu, 10 Feb 2022 09:59:52 GMT

this is my search string
index=mimecast NOT [ | inputlookup Sender1.csv | return Sender ]

the number of rows in csv is now reduced to 34000

The search result is same if do search for
index=mimecast
or
index=mimecast NOT [ | inputlookup Sender1.csv | return Sender ]

it is still not filtering or removing the email addresses listed in the csv.

Re: Is it possible if a non matching domain is emailing us, it should display in a dashboard?

ITWhisperer — Thu, 10 Feb 2022 10:44:55 GMT

Try without the return

index=mimecast NOT [ | inputlookup Sender1.csv ]

Re: Is it possible if a non matching domain is emailing us, it should display in a dashboard?

tonyxavierj — Thu, 10 Feb 2022 11:11:25 GMT

no change, it is still showing all the results.

no filtering applied.