Splunk ITSI

Splunk AI in ITSI cannot find correct thresholds or weekday based time policies

LH_Splunker
Explorer

Hi everyone, 

I've revently tested the new Splunk AI feature within Splunk ITSI to define thresholds based on historic Data/KPI points. ("Test" as in I literally created very obvious dummy-data for the AI to process and find thresholds for. Sort of Trust test of the AI really does find usuable thresholds. )

Example

Every 5 minutes the KPI takes the latest value which I've set to correspond with the current weekday (+ minimal variance)

For example: All KPI values on Mondays are within the range of 100-110, Tuesdays 200-210, Wednesdays 300-310 and so forth. 

This is a preview of the data: 

LH_Splunker_1-1736944612182.png

Now after a successful backfill of 30 days I would have expected the AI to see that each weekday needs its own time policy and thresholds. 

However the result was this: 

LH_Splunker_3-1736944864109.png

No weekdays detected, and instead it finds time policies for every 4hours regardless of days? 

By now I've tried all possible adjustments I could think of (increasing the number of data points, greater differences between data points, other algorithmn, waiting for the next in hopes it would recalibrate itself over midnight, etc.)

Hardly any improments at all and the thresholds are not usuable like this as it would not be able to detect outliers on mondays (expected values 100-110, outlier would 400 but not detected as it's still within thresholds. Thus my question to the community:

  • Does anyone have some ideas/suggestions how I could make the AI understand the simple idea of "weekly time policies" and how I could tweak it? (Aside from doing everything manually and ditching the AI-Idea as a whole)? 
  • Does anyone have good experience with Splunk AI defining Thresholds and if so what were the use cases?
Labels (3)
0 Karma
Get Updates on the Splunk Community!

Now Available: Cisco Talos Threat Intelligence Integrations for Splunk Security Cloud ...

At .conf24, we shared that we were in the process of integrating Cisco Talos threat intelligence into Splunk ...

Preparing your Splunk Environment for OpenSSL3

The Splunk platform will transition to OpenSSL version 3 in a future release. Actions are required to prepare ...

Easily Improve Agent Saturation with the Splunk Add-on for OpenTelemetry Collector

Agent Saturation What and Whys In application performance monitoring, saturation is defined as the total load ...