[math] \require{textmacros} \def\bbeta{\bf \beta} \def\vvarepsilon{\bf \varepsilon} [/math]

Consider the standard linear regression model [math]Y_i = \mathbf{X}_{i,\ast} \bbeta + \varepsilon_i[/math] for [math]i=1, \ldots, n[/math] and with the [math]\varepsilon_i \sim_{i.i.d} \mathcal{N}(0, \sigma^2)[/math]. Suppose the parameter [math]\bbeta[/math] is estimated by the ridge regression estimator [math]\hat{\bbeta}(\lambda) = (\mathbf{X}^{\top} \mathbf{X} + \lambda \mathbf{I}_{pp})^{-1} \mathbf{X}^{\top} \mathbf{Y}[/math].

• The vector of ‘ridge residuals’, defined as [math]\vvarepsilon(\lambda) = \mathbf{Y} - \mathbf{X} \hat{\bbeta}(\lambda)[/math], are normally distributed. Why?
• Show that [math]\mathbb{E}[\vvarepsilon(\lambda)] = [\mathbf{I}_{nn} - \mathbf{X} (\mathbf{X}^{\top} \mathbf{X} + \lambda \mathbf{I}_{pp})^{-1} \mathbf{X}^{\top}] \mathbf{X} \bbeta[/math].
• Show that [math]\mbox{Var}[\vvarepsilon(\lambda)] = \sigma^2 [\mathbf{I}_{nn} - \mathbf{X} (\mathbf{X}^{\top} \mathbf{X} + \lambda \mathbf{I}_{pp})^{-1} \mathbf{X}^{\top}]^2[/math].
• Could the normal probability plot, i.e. a qq-plot with the quantiles of standard normal distribution plotted against those of the ridge residuals, be used to assess the normality of the latter? Motivate.