How to use evaluate function across multiple multivalue fields

[mngeow]

Hi,

I am trying to create an anomaly detector for unusually high thruputs across all sourcetypes in my Splunk internal logs. I have used the following code to compile a table of the sourcetype by thruput rate(kilobytes/s) by the time :

index=_internal
source=*metrics.log
group=*sourcetype*
| xyseries _time,series,kbps

I am using the standard deviation method to determine my threshold to find the outliers for each sourcetype.

I am using the following code from the Splunk MLTK addon to detect my outliers:

|evenstats avg("$sourcetype$") as avg stdev("$sourcetype$") as stdev
| eval lowerBound=(avg-stdev*20),upperBound=(avg+stdev*20)
| eval isOutlier=if('$sourcetype$' < lowerBound OR '$sourcetype$' > upperBound ,1 , 0)
| where isOutlier=1


But I do not know how to calculate the average and standard deviation of the thruput rate of each sourcetype using the table generated above. I know that this can be done manually by keying in the sourcetypes. But I have over 20 sourcetypes, is there a way to make a loop using SPL that will loop through all sourcetypes and perform the relevant calculations?

Thanks!

[woodcock · Accepted solution]

This answer will give you more than you need but it has EVERYTHING (prepare to do some work) and it doesn't use the black-box "magic" of ML:

https://answers.splunk.com/answers/511894/how-to-use-the-timewrap-command-and-set-an-alert-f.html