All Apps and Add-ons

Anomalies detection using Splunk vs R

New Member

Hi all,

I have a small numerical dataset to perform anomalies detection. My data contains 177 events and I have imputed 3 records to check anomalies using Splunk and R. I applied LOF algo. in R and find out all 3 events. In splunk, I applied anomalies command and generated unexpectedness score for each event and sorted in decreasing order. I found normal events as anomalous and imputed event comes into 6 place. Even all event found as with very less unexpectedness score.

I am just curious to know whether Splunk perform well with machine learning commands such as anomalies, outliers, cluster etc. one more query i have whether splunk work well with numerical data which contains timpstamp.

Tags (1)
0 Karma

Path Finder

If the dataset is something like:

_time, value
10:00:01, 3.22
10:00:04, 32.22

An effective approach to identify anomalies is to create a statistical model of the numerical values, and computing the probability of a specific data value. If the probability is low, then the value is anomalous.

Generally, to accurately model these data and avoid false positives these models needs to be more sophisticated than a simple Normal distribution. In addition, these data are generally periodic and so the models need to allow for daily and weekly patterns.

LOF methods can be effective on static low dimensional datasets, but suffer from similar issues to kernel density functions (overfitting, linear space complexity etc.).

Further details are available here:

We have built an app to automatically identify anomalies in numeric and categorical data using these techniques:

0 Karma
Get Updates on the Splunk Community!

Introducing Edge Processor: Next Gen Data Transformation

We get it - not only can it take a lot of time, money and resources to get data into Splunk, but it also takes ...

Take the 2021 Splunk Career Survey for $50 in Amazon Cash

Help us learn about how Splunk has impacted your career by taking the 2021 Splunk Career Survey. Last year’s ...

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...