All Apps and Add-ons

Splunk Machine Learning Toolkit: Can I apply a machine learning model to external data via REST?

walterd
Engager

Hi all,

I work in a Security Operations Center for an organization, which is targeted by lots of Spam and Phishing mails.

Our users have the opportunity to report those Phishing mails to us, where we continue analysing them.
The issue is, that we get plenty of Spam mails reported too.

Now we would like to filter out only the Phishing ones.
Therefore, I have obtained a dataset and trained a model in Splunk.

Regarding the worklow, I am not sure on how to integrate Splunk in the chain.
In our current situation, the raw metadata of the extracted features cannot be streamed in realtime to Splunk but has to be uploaded manually one a week.

I am looking for a solution something like this:
1. Extract the features from newly reported mails
2. Call a ML algorithm with the previously obtained features as paramaters in Splunk via REST
3. Get the result of the classification in an instant

Can this be done within Splunk without having the data to be first forwarded and indexed?

Cheers,
Dan

1 Solution

astein_splunk
Splunk Employee
Splunk Employee

Hi

"In our current situation, the raw metadata of the extracted features cannot be streamed in realtime to Splunk but has to be uploaded manually one a week."

Why is this? Most of the customers I work with can extract their features from the streaming data in Splunk, which are used in the training of the model (|fit) . Can you elaborate?

If you go through the SDK/API documentation for Splunk, you can submit a search to splunk and get the results back from an external system. An example search can be making an empty result set (makeresults) , filling the correct fields with features (|eval) and then applying the ML model to get the result back (|apply).

View solution in original post

astein_splunk
Splunk Employee
Splunk Employee

Hi

"In our current situation, the raw metadata of the extracted features cannot be streamed in realtime to Splunk but has to be uploaded manually one a week."

Why is this? Most of the customers I work with can extract their features from the streaming data in Splunk, which are used in the training of the model (|fit) . Can you elaborate?

If you go through the SDK/API documentation for Splunk, you can submit a search to splunk and get the results back from an external system. An example search can be making an empty result set (makeresults) , filling the correct fields with features (|eval) and then applying the ML model to get the result back (|apply).

walterd
Engager

I only saw now your reply.

Thank you very much! This was exactly the solution I was looking for.

Regarding your question, we have some sort of a process bottleneck in place, which will need time to get removed.

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

Industry Solutions for Supply Chain and OT, Amazon Use Cases, Plus More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...