Mean squared prediction error

In statistics the mean squared prediction error of a smoothing or curve fitting procedure is the expected value of the squared difference between the fitted values implied by the predictive function $\widehat {g}$ and the values of the (unobservable) function g. It is an inverse measure of the explanatory power of ${\displaystyle {\widehat {g}},}$ and can be used in the process of cross-validation of an estimated model.

If the smoothing or fitting procedure has operator matrix (i.e., hat matrix) L, which maps the observed values vector $y$ to predicted values vector ${\hat {y}}$ via ${\displaystyle {\hat {y}}=Ly,}$ then

$\operatorname {MSPE}(L)=\operatorname {E}\left[\left(g(x_{i})-\widehat {g}(x_{i})\right)^{2}\right].$

The MSPE can be decomposed into two terms (just like mean squared error is decomposed into bias and variance); however for MSPE one term is the sum of squared biases of the fitted values and another the sum of variances of the fitted values:

$\operatorname {MSPE}(L)=\sum _{{i=1}}^{n}\left(\operatorname {E}\left[\widehat {g}(x_{i})\right]-g(x_{i})\right)^{2}+\sum _{{i=1}}^{n}\operatorname {var}\left[\widehat {g}(x_{i})\right].$

Note that knowledge of g is required in order to calculate MSPE exactly.

Estimation of MSPE

For the model $y_{i}=g(x_{i})+\sigma \varepsilon _{i}$ where $\varepsilon _{i}\sim {\mathcal {N}}(0,1)$, one may write

$\operatorname {MSPE}(L)=g'(I-L)'(I-L)g+\sigma ^{2}\operatorname {tr}\left[L'L\right].$

The first term is equivalent to

$\sum _{{i=1}}^{n}\left(\operatorname {E}\left[\widehat {g}(x_{i})\right]-g(x_{i})\right)^{2}=\operatorname {E}\left[\sum _{{i=1}}^{n}\left(y_{i}-\widehat {g}(x_{i})\right)^{2}\right]-\sigma ^{2}\operatorname {tr}\left[\left(I-L\right)'\left(I-L\right)\right].$

Thus,

$\operatorname {MSPE}(L)=\operatorname {E}\left[\sum _{{i=1}}^{n}\left(y_{i}-\widehat {g}(x_{i})\right)^{2}\right]-\sigma ^{2}\left(n-2\operatorname {tr}\left[L\right]\right).$

If $\sigma ^{2}$ is known or well-estimated by $\widehat {\sigma }^{2}$, it becomes possible to estimate MSPE by

$\operatorname {\widehat {MSPE}}(L)=\sum _{{i=1}}^{n}\left(y_{i}-\widehat {g}(x_{i})\right)^{2}-\widehat {\sigma }^{2}\left(n-2\operatorname {tr}\left[L\right]\right).$

Colin Mallows advocated this method in the construction of his model selection statistic C_p, which is a normalized version of the estimated MSPE:

$C_{p}={\frac {\sum _{{i=1}}^{n}\left(y_{i}-\widehat {g}(x_{i})\right)^{2}}{\widehat {\sigma }^{2}}}-n+2\operatorname {tr}\left[L\right].$

where p comes from the fact that the number of parameters p estimated for a parametric smoother is given by $p=\operatorname {tr}\left[L\right]$, and C is in honor of Cuthbert Daniel.

See also

• Mean squared error
• Errors and residuals in statistics
• Law of total variance