Machine Learning Toolkit: Has anyone used this app with data exfiltration?

[mcbradford]

Hello,

Not sure if anyone has used the Machine Learning Toolkit for data exfiltration (data exfil)? I would like to identify outliers from my email traffic. I have the message size within my data, so I was hoping to use this data to establish a baseline and alert on the outliers. Any thoughts on doing this with Splunk and/or the Machine Learning Toolkit?

[rjthibod · Accepted solution]

I have not used it for this purpose, but using the Median Absolute Deviation algorithm (MAD) under the Outlier Detection set of tools might prove useful.

MAD is more robust than using something like standard deviation, in part because it does not rely on a normal distribution assumption.

The tricky thing you would need to figure out is how to setup the model via fit in order to determine your thresholds based on certain message types or metadata (e.g., source, sender, etc.). Once you decide on what dimensions are important to differentiate message types, it should be pretty shortforward to use the toolkit to set the parameters for the populations and then setup some saved searches that would use apply.