Hi, I'm new to ML in Splunk. As a POC I'm trying to forecast expected call volumes for a service, and then alert if we are under or over the expected volume. I'm training the model on 30 minute chunks of historic data, which goes back about 7 months. Call volumes are periodic based on both the time of day and day of week, so I'd thought I would use a period of 336 (the number of half hours in a week):
| mstats sum(_value) as call_count WHERE metric_name="myServiceCalls" span=30m@w index=my_metrics | makecontinuous _time span=30m@h | fillnull value=0 call_count
| fit StateSpaceForecast "call_count" output_metadata=true holdback=1week forecast_k=2week conf_interval=50 period=336 into "service_call_count"
I am trying to experiment with using "apply" on the previous 1/2h hours of live data. Maybe "apply" is the wrong tool here.
index=myliveIndex earliest="-30m@h" latest="@h" host="p*" sourcetype="p*" "my service string"
| bin _time span="30m" aligntime="@h"
| stats count(_raw) AS call_count BY _time | apply "service_call_count"
The error I'm getting is (I believe) that I am not supplying 336 data points for the apply function:
Error in 'apply' command: holdback value equates to too many events being withheld (336 >= 2).
I now understand that apply expects to see an entire "period" of data, so I'm guessing this is the wrong approach for my usecase. Can anyone point me in the right direction? Really, I want to lookup the predicted range of counts for a given 1/2 hour and then alert when we're out of range.
