Alerting

where to start with adaptive thresholding and machine learning

HattrickNZ
Motivator

Below is a pic of a kpi put ontop of each other over 4 weeks.
so 1 line, in the graph, is 1 weeks of data at 5minutes resolution.
X, in the graph, is what should have happened - the black line
Y, in the graph, is what did happen - the purple line.

I am exploring way to trigger on this type of behaviour, but I don't really have a clue where to start.

Is there a way I can input my search that produces the below graph, and see how some algorithim or adaptive thresholding would have possible picked this up?

Appreciat any advice in this area.

alt text

Further exploring:
https://answers.splunk.com/answers/593406/how-to-get-adaptive-threshold-without-using-itsi-f.html
https://answers.splunk.com/answers/474759/where-can-i-find-information-about-adaptive-thresh-1.html

EDIT1

I may need to post a new question but I am thinking of using the outliers chart but it is not showing all the fields as per the serach:

... | fields + _time kpi1_lastWeek lowerBand upperBand isOutlier kpi1_thisWeek *

The idea is that upperBand and lowerBand are derived from kpi1_lastWeek and the Bands create a threshold in which if the kpi1_thisWeek breaches the Bands then there would be an alert/trigger. But I am not sure if I can do this with outliers Chart.
So my question now is how many fields can I have in an outliers chart? And can I have an appendcols in the search because I will have 2 searches, 1 for this week and 1 for last week?
note: There maybe a better way than usingappendcols maybe stats latest(kpi1) earliest(kpi1) (need to check)????

0 Karma

jwiedemann_splu
Splunk Employee
Splunk Employee
0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...