Hi @woodcock ,
Thanks for your reply. I completely agree that the order of search time operations is what is causing us issues, although I'm still unsure if what you've suggested will combat the problem.
Please correct me if I'm wrong, but if the calculated field is created before the tags, won't the search be the same?
My correlation search is below, which may add some more context to the problem.
| tstats allow_old_summaries=true sum(All_Email.recipient_count) as count,dc(All_Email.dest) as dest_count from datamodel=Email.All_Email where (All_Email.src_category!="email_servers" OR All_Email.src_category!=* AND All_Email.directionality="originating" AND NOT All_Email.source_id="STOREDRIVER") by "All_Email.src" ,_time span=1h | drop_dm_object_name("All_Email") | xswhere count from recipients_by_src_1h in email is above high OR dest_count from destinations_by_src_1h in email is above high
So from my understanding, even if I added the tags in this search, the rule would still alert because the calculated field "All_Email.recipient_count" will count the fields with the items we want to omit.
If I added the tag NOT tag=STOREDRIVER to the constraint field in the data-model (even though I know you suggested against it earlier) this will then remove the STOREDRIVER completely which will then give us a more accurate representation of the emails.
To further expand on this, when an alert appears in ES stating that there were 193 SMTP connections were observed, when taking out STOREDRIVER there are only 39 actual emails were sent, the STOREDRIVER events are actually a Microsoft Exchange process. So if we take them out of the data model completely, then recipient_count should give us a more accurate count of SMTP connections made.
... View more