Incorrect, the objective is VendorDomain1/2/3 emails in.. they will be filter.routeDirection=inbound with their domain as src_user Then.... DomainA emails out... to VendorDomain 1/2/3 they will be filter.routeDirection=outbound with their domain as src_user too... This is because regardless of direction, there are fields mapped as recipient/src_user per the CIM for DataModel EMAIL... which is really really well mapped in this case... so using even the the base EMAIL Data Model, it will map.. Proofpoint also give you a Data Model called "Proofpoint On Demand Email Security" So know we can also borrow from this concept as well. The end goal here @ITWhisperer is that we are looking at the emails coming in, and if we reply to them. Sadly the values of src_user, and recipient are the same no matter the direction. However, when I ran the below for 90 days... i only got 3 domains back which told me who of OUR domains emailed out of our Email Gateway ... i think my append is trash.... Here's my current search using part of your logic... index=email sourcetype=pps_messagelog src_user!="" recipient!="" filter.routeDirection!=Internal recipient!=*ppops.net* (final_rule=pass OR final_rule=outbound_virus_clean)
| transaction msg.header.message-id
| rename envelope.rcpts{} as recipient, filter.routeDirection AS direction
| eval recipient=lower(recipient)
| mvexpand src_user
| mvexpand recipient
| makemv delim="([^,]+),?" allowempty=false recipient
| makemv delim="([^,]+),?" allowempty=false src_user
| rex field=recipient (.*@.*\.(?<recipient>.*\..*)$)
| rex field=recipient .*@(?<recipient2>.*\..*)$
| rex field=src_user (.*@.*\.(?<src_user>.*\..*)$)
| rex field=src_user .*@(?<src_user2>.*\..*)$
| eval domain=mvappend(src_user2,recipient2)
| chart count by domain direction
| where inbound > 200 AND outbound > 90 Explanation index=email with sourcetype don't give blank src_user or recipient. block the visibility of internal or when recipient is ending in ppops.net for bounce backs/digests then here I ask for all final_rule=pass or outbound_virus_clean which indicates to the gateway that it didn't go to any quarantine in/out... those are the rules on a good email both in and out. renames of two fields... then we start to do our break out... and just for good measure, I expand both... then rex them using the various rex's then I append values of the src_user2 and recipient2 as they are the best and have ONLY the domains... then using your chart count, then I did a thing where the people emailing us is 200 and our replies back in 90 days was 30% of that... or at least 90 emails back... I will eventually output this to a list and maybe run another check on it for clarity, afterwards dump this as a lookup list that can be fed various places (Threat intel, correlation rules, playbooks, etc) to then use as source of truth so to speak on "This is a vendor"... For those interested, and I know this is long winded... but the Data Model for Proofpoint is quite good.. _time host=(IDM/HF) source (proofpoint_message_log) action_dkimv.rule (output of the rule that impacted the result of the sending email dkim rule) action_dmarc.rule (output of the rule that impacted the result of the sending email dmarc rule) action_spf.rule (output of the rule that impacted the result of the sending email spf rule) connection.tls.inbound.version (what version was used to TLS on all inbound emails) filter.actions.action (all actions taken on the mail) filter.actions.isFinal (what was the final action) filter.actions.module (what modules were used to give the actions/rule) filter.actions.rule (what rules were applied to the mail) filter.disposition (what was the ruling of the email based on decision tree) filter.quarantine.folder (if it went to quarantine, what folder was it) filter.routeDirection (what direction was it? Internal, Inbound, or Outbound) final_action (what was the final action PERIOD) final_module (what was the last module that it went through) final_rule (was was the final rule PERIOD) HINT: it maps directly to the RULES of each module is_encrypted (was the email encrypted TRUE/FALSE) msg.header.message-id (the message ID) msg.header.subject (Better to always look at as logic in parser takes care of normalization of html subjects) msg.header.to (who was the email to IS MULTIVALUED) msg_header_from (who sent the email)
... View more