[math] \require{textmacros} \def \bbeta {\bf \beta} \def\fat#1{\mbox{\boldmath$#1$}} \def\reminder#1{\marginpar{\rule[0pt]{1mm}{11pt}}\textbf{#1}} \def\SSigma{\bf \Sigma} \def\ttheta{\bf \theta} \def\aalpha{\bf \alpha} \def\ddelta{\bf \delta} \def\eeta{\bf \eta} \def\llambda{\bf \lambda} \def\ggamma{\bf \gamma} \def\nnu{\bf \nu} \def\vvarepsilon{\bf \varepsilon} \def\mmu{\bf \mu} \def\nnu{\bf \nu} \def\ttau{\bf \tau} \def\SSigma{\bf \Sigma} \def\TTheta{\bf \Theta} \def\XXi{\bf \Xi} \def\PPi{\bf \Pi} \def\GGamma{\bf \Gamma} \def\DDelta{\bf \Delta} \def\ssigma{\bf \sigma} \def\UUpsilon{\bf \Upsilon} \def\PPsi{\bf \Psi} \def\PPhi{\bf \Phi} \def\LLambda{\bf \Lambda} \def\OOmega{\bf \Omega} [/math]

Consider the linear regression model [math]\mathbf{Y} = \mathbf{X} \bbeta + \vvarepsilon[/math] with [math]\vvarepsilon \sim \mathcal{N} ( \mathbf{0}_n, \sigma^2 \mathbf{I}_{nn})[/math]. This model is fitted to data, [math]\mathbf{X}_{1,\ast} = (4, -2)[/math] and [math]Y_1 = 10[/math], using the lasso regression estimator [math]\hat{\bbeta}(\lambda_1) = \arg \min_{\bbeta} \| Y_1 - \mathbf{X}_{1,\ast} \bbeta \|_2^2 + \lambda_1 \| \bbeta \|_1[/math].

• How many nonzero elements does the lasso regression estimator with an arbitrary [math]\lambda_1 \gt 0[/math] have for these data?
• Ignore the second covariate and evaluate the lasso regression estimator for [math]\lambda_1=8[/math].
• Suppose that, when regressing the response on each covariate separately, the corresponding lasso regression estimates with [math]\lambda_1=8[/math] are [math]\hat{\beta}_1 (\lambda_1) = 2 \tfrac{1}{4}[/math] and [math]\hat{\beta}_2 (\lambda_1) = -4[/math]. Now consider the regression problem with both covariates in the model. Does the lasso regression estimate with [math]\lambda_1=8[/math] then equal [math]\hat{\bbeta} (\lambda_1) = (2 \tfrac{1}{4}, 0)^{\top}[/math], [math]\hat{\bbeta} (\lambda_1) = (0, -4)^{\top}[/math], or some other value? Motivate!