Using Machine Learning for Hunting Security Threats

[melissap]

Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for AI / Machine Learning based Analytics to supercharge threat detection and minimize the operational overheads of maintaining conventional static detection rules in large scale SOC. However, use of AI / Machine Learning in Security Operations is challenging due to the complex cyber security big data and numerous attacker techniques. 

In this webinar, Muath Saleh and Hafiz Farooq (from Saudi Aramco) shall explain how to use the analytical power of Splunk to hunt for cyber and insider threats, and also utilizes the Splunk Machine Learning Toolkit (MLKT) for novelty and outlier detection from the noisy security datasets. This webinar purviews Saudi Aramco’s experience of using Splunk for handling security big data, and explains amazing key capabilities for effective operational security procedures and threat hunting.

[melissap]

Here is the Q&A that happened during the live Tech Talk. 

Q: Hi, I am a scikit expert, can I import all scikit algorithms in Splunk?

A: Yes, since scikit and splunk both support python based platform, therefore it is very easy to import all algorithms if you are having some splunk development background. Also checkout the deep learning toolkit for examples + juniper notebook integrations: https://splunkbase.splunk.com/app/4607 

Q: How did you select these ML use cases. based on some standard or guideline?

A: We use machine learning algorithms for use cases where security data is very noisy and you can't make static threat rules based on heuristics.

Q: Does splunk machiine learning require any license? like Elastic?

A: No additional license needed.

Q: What's the performance impact on Splunk when we use machine learning

A: mostly on CPU and Memory. it depends what your ML model is and how often you let it re-train. Since Splunk MLTK can be scheduled instead of real time, so it all depends how you want to handle the workload. Also Splunk is quite scalable and therefore there is always an option scale splunk indexing layer to support your ML operations without any licensing issues.

Q: How can we ensure that training data doesn't contain any malicious events which might negate the ML model that is derived from training data ?

A: You can watch this webinar: https://events.splunk.com/Data-Science-Advances-in-Cyber-Security 

Q: Can Splunk help us in developing machine learning use cases. Is any professional services support available for ML use cases? I think there are not many skilled professionals available in the industry who can actually develop data science use cases?

A: Hi, we do a more intro session mid December with many tips and guides - how to use it out of the box and how to go on the path establishing skills.

Q: is there any effect or consequences to be consider on the production environments while using MLTK ?

A: Hi, yes - there will be more processing power utilized. this depends on which ML models you run, how often you teach it new etc.

Q: In splunk does ML algorithm built in? or should we build it from scratch?

A: You can do both - especially it's key to understand them even they are out of the box. Like for security many pre-build once are in the security essentials app embedded. The MLTK and Depp Learning Toolkit include more data scientific examples.

Q: Want to ask how to start with ML in Splunk ? Can you share a Roadmap?

A: checkout what the colleagues from Yapi Credit established from no ML skills to 1 year: https://conf.splunk.com/watch/conf-online.html?search=SEC1471B#/ 

Q: What is your view on using deep learning and using time as a parameter input

A: Splunk Deep Learning toolkit also provide to do deep learning analysis on your security data. But since security data is quite complex, multi-featured and complex therefore it's less effective in comparison to machine learning. But there is no harm in performing DL based anomaly detection and novelty threat detection. Thanks

Q: Can you please share the list of Use cases. Implemented already, instead of just examples? 

A: You can find the once shared back from the Splunk security research team in the security essential app. Those once you have seen today are also in place - that's why we wanted to bring this webinar with real world experience to you. 😉 hope that helps.

[melissap]

Here are additional resources to help you continue on your journey.

Splunk Machine Learning Toolkit

• https://splunkbase.splunk.com/app/2890

Splunk App for Data Science and Deep Learning

• https://splunkbase.splunk.com/app/4607

Whitepaper: Operationalize Machine Learning to Find Malicious Domains

• https://www.splunk.com/en_us/form/operationalizing-machine-learning-to-detect-malicious-domain/thank...

Blog: Out of the Box Machine Learning Use Cases in Splunk Enterprise Security

• https://www.splunk.com/en_us/blog/security/new-machine-learning-in-splunk-enterprise-security-conten... 

Webinar: ML in Security - Risky SPL Detection with MLTK

• https://events.splunk.com/ML-in-Security-Risky-SPL-Detection-with-MLTK

Webinar: Prevent Data Downtime with MLTK

• https://events.splunk.com/Prevent-Data-Downtime-with-Machine-Learning

[arajoot]

nice one, very informative

[prunellapassare]

As a newbie aboard, I'm so glad to find this, thanks! I'm an engineering and programming science student, so I find using machine learning for hunting security threats fascinating. This surely can help security professionals to detect and respond to threats faster and more effectively. However, I also think that it's important for students to learn about the limitations of machine learning and the potential ethical concerns that can arise from its use in security. On https://papersowl.com/write-my-essay-please I find many interesting ideas from academic experts on this topic. Frequently I ask them to write my essay to gain valuable knowledge. Also, I consider students should learn about other strategies for improving security, such as user education and awareness, secure coding practices, and vulnerability management. I'm sure by developing a well-rounded understanding of security, we - students - can contribute to developing more effective and ethical security solutions.