Hi @TylerJVitale,
So far this works:
| makeresults
| eval hosts_predict=split("host1,host2,host3,host4,host5", ",")
| mvexpand hosts_predict
| map maxsearches=5 search="search index=\"index_to_search_in\" latest=\"-0d@d\" host=\"$hosts_predict$\" | table _time host VOLUME | bin _time span=1d | stats sum(VOLUME) as sum_VOLUME by _time host | predict sum_VOLUME as prediction algorithm=\"LLP5\" future_timespan=\"30\" holdback=\"14\" period=7 lower\"95\"=lower\"95\" upper\"95\"=upper\"95\" | filldown host"
| eval isOutlier=if(sum_VOLUME < 'lower95(prediction)' OR sum_VOLUME > 'upper95(prediction)', 1, 0)
| where isOutlier=1
| fields - isOutlier
A little explaining:
In the makeresults command you are selecting the hosts that you would like to run the predict command to. This could either be a lookup or a list of hosts that results from another search.
The map command then takes each row as input to feed the predict search. This search is where the data to feed the analysis comes from; it just changes the host that it is applied to. Highly recommended to be fed with some form of summary or acelerated data since; depending on your setup, this could take very long and consume a lot of resources.
The last eval command is the Outlier detection from your original search.
I agree with @grana_splunk that is highly recommended to evaluate another way to accomplish the outlier detection logic. Here I present you with several alternatives:
Run a report to generate a lookup which contains per-day-basis-threshold (customizable) per host and compare it with your current data. This highly reduces the overhead of performing the predict command in an alert since it is a simple lookup operation. (I have applied this in many scenarios and works great)
As mentioned by @grana_splunk, use the DensityFunction algorithm in the MLTK 4.2.
For algorithms to detect outliers you could use IQR (Interquartile range) or Standard Deviation. You should check how Splunk ITSI applies some of this procedures to generate Adaptive Thresholds. https://www.splunk.com/blog/2018/01/16/ensuring-success-with-itsi-threshold-and-alert-configurations-part-2-adaptive-thresholding.html
To replace the makeresults you could do the following:
index=\"index_to_search_in\"
| table host
| dedup host
| rename host as hosts_predict
| map ...
With a lookup:
| inputlookup list_of_hosts.csv
| field host
| rename host as hosts_predict
| map ...
Hope it helps
... View more