Archive
Highlighted

Best approach to remove false positives (email) from search when it contains 'error'

Builder

i have a search looking for "error" OR "fail" OR "failed" OR "exception" in events

However we are picking up false positives where there is an email in a field such as:

mr.error@hotmail.com

Also the position of the email is in different places within the field.

How best to exclude ?

sample fields:

msg:     LogCorrelationId XXXXXXXXXX. Email mr.error@hotmail.com. Info: Authentication MyAccountRegistrationStarted

msg:     2019-04-01T00:48:48.828Z facebook|XXXXX::Rules::EagerLinking:: searching for other users with email: [ 'mr.error@hotmail.com', 'mr.error@hotmail.com' ]    

msg:     2019-04-01T00:48:42.535Z ::identify-user-otp:: IsOTP: [{"name":"Mr X","email":"mr.error@hotmail.com","given_name":
Tags (1)
0 Karma
Highlighted

Re: Best approach to remove false positives (email) from search when it contains 'error'

Champion

Since "NOT" will slow down the search, I think that it is good to extract and filter once.

(your search) "error" OR "fail" OR "failed" OR "exception" |search NOT ("*error@*.*" OR "error*@*.*")
0 Karma
Highlighted

Re: Best approach to remove false positives (email) from search when it contains 'error'

Builder

Thanks Hiroshi - this looks ok for error - but trying to future-proof there may potentially be emails that also have the other keywords in also.

0 Karma
Highlighted

Re: Best approach to remove false positives (email) from search when it contains 'error'

Champion

I think that the condition(contains ) should be a lookup file.

 |search NOT [|inputlookup your_lookup.csv|table contains |rename contains as query]
0 Karma
Highlighted

Re: Best approach to remove false positives (email) from search when it contains 'error'

Builder

hi Hiroshi - are you suggesting we have a lookup with all emails in ? i don't think thats possible to get a list of all potentially tens of thousands of emails ?

0 Karma