All Apps and Add-ons

R squared negative

diasna1
New Member

During the development of a project, we faced some problems concerning the output of correlation coefficient R^2, when using the Splunk Machine Learning Toolkit version 3.1.0.

About the results concerning the regression coefficient, when evaluating the performance of a fitted time series, as described here below, we obtain a negative R^2.
alt text

If we consider the R^2 as:

R^2=SSR/SST

where
SSR=sum_i (y_i-hat{y_i})^2 stands for Regression Sum of Squares and
SST=sum_i (y_i-overline{y})^2 for Total Sum of Squares,
it’s impossible to obtain negative values of it. But we are not sure how the algorithm calculates it.
We suspect that it’s being considered the adjusted R^2, since this measure may have a negative value in some particular situations. However, we are not sure of it and how do they extend the result to time series. Also, we haven’t found any information about it so far.
We hope that somebody can help us figure out what’s going on 🙂
All the best with your Splunk queries and work!

0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

Hello @diasna1 - here is how we actually calculate the R^2 value as of MLTK 3.2.0:

| rename "$a$" as _actual, "$p$" as _predicted 
| eventstats avg(_actual) as _avgActual 
| eval _actualMinusAvg = _actual - _avgActual, _residual = _actual - _predicted 
| stats sumsq(_actualMinusAvg) as _sumsqActualMinusAvg, sumsq(_residual) as _sumsqResidual, count(_residual) as _sampleCount 
| eval rSquared = round(1 - _sumsqResidual / _sumsqActualMinusAvg, 4), RMSE = round(sqrt(_sumsqResidual / _sampleCount), 2) 
| table rSquared RMSE

alt text

Please feel free to review the similar definition on wikipedia, and let me know if I made a mistake.

https://en.wikipedia.org/wiki/Coefficient_of_determination#Definitions

So to answer your question directly, R^2 can ben negative whenever SSR is greater than SST.

Additionally, you may find this cross validated question & answer relevant: https://stats.stackexchange.com/a/12991/78566

0 Karma

walkerhound
Path Finder

When you hover to see what the R squared means, it says "The square of the correlation coefficient..." This would seem to imply that R squared would be positive. Should the information under the hover be changed?

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...