Alerting

where to start with adaptive thresholding and machine learning

HattrickNZ
Motivator

Below is a pic of a kpi put ontop of each other over 4 weeks.
so 1 line, in the graph, is 1 weeks of data at 5minutes resolution.
X, in the graph, is what should have happened - the black line
Y, in the graph, is what did happen - the purple line.

I am exploring way to trigger on this type of behaviour, but I don't really have a clue where to start.

Is there a way I can input my search that produces the below graph, and see how some algorithim or adaptive thresholding would have possible picked this up?

Appreciat any advice in this area.

alt text

Further exploring:
https://answers.splunk.com/answers/593406/how-to-get-adaptive-threshold-without-using-itsi-f.html
https://answers.splunk.com/answers/474759/where-can-i-find-information-about-adaptive-thresh-1.html

EDIT1

I may need to post a new question but I am thinking of using the outliers chart but it is not showing all the fields as per the serach:

... | fields + _time kpi1_lastWeek lowerBand upperBand isOutlier kpi1_thisWeek *

The idea is that upperBand and lowerBand are derived from kpi1_lastWeek and the Bands create a threshold in which if the kpi1_thisWeek breaches the Bands then there would be an alert/trigger. But I am not sure if I can do this with outliers Chart.
So my question now is how many fields can I have in an outliers chart? And can I have an appendcols in the search because I will have 2 searches, 1 for this week and 1 for last week?
note: There maybe a better way than usingappendcols maybe stats latest(kpi1) earliest(kpi1) (need to check)????

0 Karma

jwiedemann_splu
Splunk Employee
Splunk Employee
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...

Design, Compete, Win: Submit Your Best Splunk Dashboards for a .conf26 Pass

Hello Splunkers,  We’re excited to kick off a Splunk Dashboard contest! We know that dashboards are a primary ...

May 2026 Splunk Expert Sessions: Security & Observability

Level Up Your Operations: May 2026 Splunk Expert Sessions Whether you are refining your security posture or ...