Re: Mathematica calculates RSquared wrongly?

*To*: mathgroup at smc.vnet.net*Subject*: [mg112787] Re: Mathematica calculates RSquared wrongly?*From*: Ray Koopman <koopman at sfu.ca>*Date*: Thu, 30 Sep 2010 04:53:00 -0400 (EDT)

On Sep 29, 7:48 am, Darren Glosemeyer<darreng at wolfram.com> wrote: > On 9/29/2010 3:15 AM, Ray Koopman wrote: >> On Sep 28, 3:09 am, Darren Glosemeyer<darreng at wolfram.com> wrote: >>> On 9/27/2010 4:47 AM, Lawrence Teo wrote: >>>> [...] >>>> nlm = NonlinearModelFit[sbbBN, a*x^2 + b*x + c, {a, b, c}, x] >>>> nlm["RSquared"] >>>> >>>> The RSquared by Mathematica is 0.963173 >>>> Meanwhile, Excel and manual hand calculation show that R^2 should >>>> be equal to 0.7622. >>>> >>>> Is Mathematica wrong? Thanks! >>> >>> This is as designed. For nonlinear models, the corrected (i.e. with >>> the mean subtracted out) sum of squares is sometimes used. This is >>> consistent with comparing to a constant model, but most nonlinear >>> models do not include a constant in an additive way. For this reason, >>> NonlinearModelFit uses the uncorrected (i.e. without subtracting out >>> the mean) sum of squares. >> >> This information should be included in the "Goodness-of-Fit Measures" >> section of the NonlinearModelFit documentation, which should also >> point out that RSquared is computed as 1 - (Residual SS)/(Total SS), >> and that in nonlinear models this is generally different from the >> ratio (Model SS)/(Total SS) that is sometimes cited -- e.g., >> http://reference.wolfram.com/mathematica/RegressionCommon/ref/RSquared.html >> -- as the definition of RSquared. > > The RegressionCommon documentation is for a now obsolete standard > package. The "RSquared" property for nonlinear models is described > near the bottom of > > http://reference.wolfram.com/mathematica/tutorial/StatisticalModelAnalysis.html > > The current statement is: > > "The coefficient of determination "RSquared" is the ratio of the model > sum of squares to the total sum of squares." > > I will modify this to mention that the total is the uncorrected total > for the next version. Also, the n-1 in the formula for AdjustedRSquared should be n, because the total sum of squares is uncorrected. However, all that misses the main point I was trying to make, which is that simply changing from corrected to uncorrected sums of squares will not give 1 - SS_res/SS_tot, which is how NonlinearModelFit calculates RSquared. The reason is that the residuals are not generally orthogonal to the fitted values, so the decomposition SS_tot = SS_fit + SS_res that holds for linear models does not generally hold for nonlinear models. For instance, using the data and model from the "Goodness-of-Fit Measures" section of the NonlinearModelFit documentation, the fitted values are {13.658, 2.00568, 1.48485, 14.8951, 5.6088, 10.1695, 11.0627, 5.77841, 4.51702, 5.67666, 13.4947, 11.4323}, and the residuals are {0.742037, 9.09432, 5.01515, -3.79512, 1.1912, 0.930521, 1.3373, 3.12159, 4.08298, 5.72334, -1.69468, -0.532303}. Their uncentered inner product is 50.2468; centering gives -159.435.