Data Exfiltration via E-Mail

[Skinny]

Hey everyone,

I am currently trying to write a search that monitors outgoing E-Mail traffic. The goal is to see if business-relevant information is being exfiltrated via E-Mail. Since I am new to writing SPL I tried the following:

First, I wanted to write a simple search that would show me all E-Mails where the size of the E-Mail is exceeding a set threshold. That's what I came up with:

| datamodel Email search
| search All_Email.src_user="SOMETHING I USE TO MAKE SURE THE TRAFFIC IS GOING FROM INTERNAL TO EXTERNAL" AND sourcetype="fml:*"
| stats
 values(_time) as _time
 values(All_Email.src_user) as src_user
 values(All_Email.recipient) as recipient
 values(All_Email.file_name) as file_name
 values(All_Email.subject) as subject
 values(All_Email.size) as size
 by All_Email.message_id
| eval size_MB=round(size/1000000,3)
| `ctime(alert_time)`
| where 'size_MB'>X
| fields - size

As far as I can see, it does what I initially wanted it to do.

Upon further testing and thinking, I noticed a flaw. If Data is exfiltrated over a given time through many different E-Mails, that search would not trigger since the threshold X would not be exceeded in one E-Mail. That's why I wanted to write a new Search using tstats (since the above search was pretty slow) where the traffic from A to the same recurring recipient is being added up in a given time period. If the accumulated traffic would exceed a given threshold, the search would trigger.

I then came up with this:

| tstats
min(_time) as alert_time
max(_time) as end_time
values(All_Email.file_name) as file_name
values(All_Email.subject) as subject
values(All_Email.size) as size
from datamodel=Email
WHERE All_Email.src_user="SOMETHING I USE TO MAKE SURE THE TRAFFIC IS GOING FROM INTERNAL TO EXTERNAL" AND sourcetype="fml:*"
by All_Email.src_user, All_Email.recipient
| eval size_MB=round(size/1000000,3)


This search is not finished (threshold missing, etc.) since I noticed that an E-Mail with multiple attachments does not calculate the size correctly. It lists all the sizes of the different attachments but does not calculate a sum. I think the "by All_Email.src_user, All_Email.recipient" statement does not work as I intended it to.

I would be happy to get some feedback on how to improve. Maybe the Code I wrote is way to complicated or does not work as it's supposed to. 

Since I am new to writing SPL, are there any standards on how to write clean SPL or any resources where I can study many different (good) searches so that I can improve in writing my own searches? I would appreciate any form of help!

Thank you very much!

[livehybrid]

Hi @Skinny 

I think you probably meant to use sum(All_Email.size) as size instead of values(All_Email.size) as size? Then it should sum the sizes rather than return a list.

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

[Skinny]

Hey @livehybrid ,

Thank you very much; that solved the problem!

Now that it can calculate the sum of the attachments, how do I make sure that the search accumulates every event where User A sends to the same recipient and calculates the sum of the overall traffic generated? Since I don't know how to put it to words properly, here's an example:

E-Mail 1: from User A -> to User B with size=10MB - Was sent at 11:10
E-Mail 2: from User A -> to User B with size=8MB - Was sent at 12:14
E-Mail 3: from User A -> to User C with size=20MB - Was sent at 13:41
E-Mail 4: from User A -> to User B with size=23MB - Was sent at 13:55

As shown above, user A sent to two different recipients (B and C). I now want the search to sum up the overall traffic from A to recipient X over the span of 4 hours, like so: 

Traffic of A to B = 41MB
Traffic of A to C = 20MB

Let's say the threshold of my search would be 40MB over the span of 4 hours. Could you also help me with that?

Thank you very much so far!

[livehybrid]

Hi @Skinny 

What does your search look like so far? If you're doing 

| stats sum(All_Email.size) as size by All_Email.src_user, All_Email.recipient

Then I think it should already be grouping it like this?

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will