exercise:6c18916e63: Difference between revisions

From Stochiki
(Created page with "<div class="d-none"> <math> \require{textmacros} \def \bbeta {\bf \beta} \def\fat#1{\mbox{\boldmath$#1$}} \def\reminder#1{\marginpar{\rule[0pt]{1mm}{11pt}}\textbf{#1}} \def\SS...")
 
mNo edit summary
 
Line 66: Line 66:
</math>
</math>


Substitute the <math>3^{\mbox{{\tiny rd}}}</math> order Taylor approximation into the estimating equations and use the data of [[guide:E1e82a147e#table.data4logisticExercise |Table]] to evaluate its roots using the <TT>polyroot</TT>-function (of the <TT>base</TT>-package).  
Substitute the <math>3^{\mbox{{\tiny rd}}}</math> order Taylor approximation into the estimating equations and use the data of [[#table.data4logisticExercise |Table]] to evaluate its roots using the <TT>polyroot</TT>-function (of the <TT>base</TT>-package).  
Do the same for the <math>1^{\mbox{{\tiny st}}}</math> and <math>2^{\mbox{{\tiny nd}}}</math> order Taylor approximations of <math>\exp (\beta) [ 1 + \exp (  \beta)]^{-1}</math>. Compare these approximate estimates to the one provided by the <TT>ridgeGLM</TT>-function (of the <TT>porridge</TT>-package, <ref name="vWie2021R">van Wieringen, W. and Aflakparast, M. (2021).''porridge: Ridge-Type Estimation of a Potpourri of Models''.R package version 0.2.1</ref>).
Do the same for the <math>1^{\mbox{{\tiny st}}}</math> and <math>2^{\mbox{{\tiny nd}}}</math> order Taylor approximations of <math>\exp (\beta) [ 1 + \exp (  \beta)]^{-1}</math>. Compare these approximate estimates to the one provided by the <TT>ridgeGLM</TT>-function (of the <TT>porridge</TT>-package, <ref name="vWie2021R">van Wieringen, W. and Aflakparast, M. (2021).''porridge: Ridge-Type Estimation of a Potpourri of Models''.R package version 0.2.1</ref>).
</li>
</li>
</ul>
</ul>
In the remainder consider the <math>1^{\mbox{{\tiny st}}}</math> order Taylor approximated ridge logistic regression estimator: <math>\hat{\beta}_{1^{\mbox{{\tiny st}}}} ( \lambda ) = (\lambda + \tfrac{1}{4} n)^{-1} (c - \tfrac{1}{2} n)</math>.
In the remainder consider the <math>1^{\mbox{{\tiny st}}}</math> order Taylor approximated ridge logistic regression estimator: <math>\hat{\beta}_{1^{\mbox{{\tiny st}}}} ( \lambda ) = (\lambda + \tfrac{1}{4} n)^{-1} (c - \tfrac{1}{2} n)</math>.
<ul style="list-style-type:lower-roman"><li> Find an expression for <math>\mathbb{E} [ \hat{\beta}_{1^{\mbox{{\tiny st}}}} (\lambda) ]</math>.
<ul style="list-style-type:lower-roman"><li> Find an expression for <math>\mathbb{E} [ \hat{\beta}_{1^{\mbox{{\tiny st}}}} (\lambda) ]</math>.
</li>
</li>
Line 77: Line 78:
<li> Combine the answers of parts ''d)'' and ''e)'' to find an expression for <math>\mbox{MSE} [ \hat{\beta}_{1^{\mbox{{\tiny st}}}} (\lambda) ]</math>.  
<li> Combine the answers of parts ''d)'' and ''e)'' to find an expression for <math>\mbox{MSE} [ \hat{\beta}_{1^{\mbox{{\tiny st}}}} (\lambda) ]</math>.  
</li>
</li>
<li> Plot this MSE against <math>\lambda</math> for the data provided in [[#table.data4logisticExercise |Table]] and reflect on analogue of [[#theo.Theobald2 |Theorem]] for the ridge logistic regression estimator.
<li> Plot this MSE against <math>\lambda</math> for the data provided in [[#table.data4logisticExercise |Table]] and reflect on analogue of [[guide:61c19b77c3#theo.Theobald2 |Theorem]] for the ridge logistic regression estimator.
</li>
</li>
</ul>
</ul>

Latest revision as of 01:09, 25 June 2023

[math] \require{textmacros} \def \bbeta {\bf \beta} \def\fat#1{\mbox{\boldmath$#1$}} \def\reminder#1{\marginpar{\rule[0pt]{1mm}{11pt}}\textbf{#1}} \def\SSigma{\bf \Sigma} \def\ttheta{\bf \theta} \def\aalpha{\bf \alpha} \def\ddelta{\bf \delta} \def\eeta{\bf \eta} \def\llambda{\bf \lambda} \def\ggamma{\bf \gamma} \def\nnu{\bf \nu} \def\vvarepsilon{\bf \varepsilon} \def\mmu{\bf \mu} \def\nnu{\bf \nu} \def\ttau{\bf \tau} \def\SSigma{\bf \Sigma} \def\TTheta{\bf \Theta} \def\XXi{\bf \Xi} \def\PPi{\bf \Pi} \def\GGamma{\bf \Gamma} \def\DDelta{\bf \Delta} \def\ssigma{\bf \sigma} \def\UUpsilon{\bf \Upsilon} \def\PPsi{\bf \Psi} \def\PPhi{\bf \Phi} \def\LLambda{\bf \Lambda} \def\OOmega{\bf \Omega} [/math]

Consider the logistic regression model to explain the variation of a binary random variable by a covariate. Let [math]Y_i \in \{ 0, 1\}[/math], [math]i=1, \ldots, n[/math], represent [math]n[/math] binary random variables that follow a Bernoulli distribution with parameter [math]P(Y_i = 1 \, | \, X_i) = \exp(X_i \beta) [1+ \exp(X_i \beta)][/math] and corresponding covariate [math]X_i \in \{ -1, 1 \}[/math]. Data [math]\{ (y_i, X_i) \}_{i=1}^n[/math] are summarized as a contingency table (Table):

[math]y_i=\texttt{0}[/math] [math]y_i=\texttt{1}[/math]
[math]X_i=\texttt{-1}[/math] 301 196
[math]X_i= \texttt{\textcolor{white}{-}1}[/math] 206 297


The data of Table is only to be used in parts [math]''c)''[/math] and [math]''g)''[/math].

  • Show that the estimating equation of the ridge logistic regression estimator of [math]\bbeta[/math] is of the form:
    [[math]] \begin{eqnarray*} c - n \exp (\beta) [ 1 + \exp ( \beta)]^{-1} - \lambda \beta & = & 0, \end{eqnarray*} [[/math]]
    and specifiy [math]c[/math].
  • Show that [math]\hat{\beta} (\lambda) \in (\lambda^{-1} (c-n), \lambda^{-1} c)[/math] for all [math]\lambda \gt 0[/math]. Ensure that this is a meaningful interval, i.e. that it is non-empty, and verify that [math]c \gt 0[/math]. Moreover, conclude from the interval that [math]\lim_{\lambda \rightarrow \infty} \hat{\beta} (\lambda) = 0[/math].
  • The Taylor expansion of [math]\exp (\beta) [ 1 + \exp ( \beta)]^{-1}[/math] around [math]\beta=0[/math] is:
    [[math]] \begin{eqnarray*} \exp (\beta) [ 1 + \exp ( \beta)]^{-1} & = & \tfrac{1}{2} + \tfrac{1}{4} x - \tfrac{1}{48} x^3 + \tfrac{1}{480} x^5 + \mathcal{O} (x^6). \end{eqnarray*} [[/math]]
    Substitute the [math]3^{\mbox{{\tiny rd}}}[/math] order Taylor approximation into the estimating equations and use the data of Table to evaluate its roots using the polyroot-function (of the base-package). Do the same for the [math]1^{\mbox{{\tiny st}}}[/math] and [math]2^{\mbox{{\tiny nd}}}[/math] order Taylor approximations of [math]\exp (\beta) [ 1 + \exp ( \beta)]^{-1}[/math]. Compare these approximate estimates to the one provided by the ridgeGLM-function (of the porridge-package, [1]).

In the remainder consider the [math]1^{\mbox{{\tiny st}}}[/math] order Taylor approximated ridge logistic regression estimator: [math]\hat{\beta}_{1^{\mbox{{\tiny st}}}} ( \lambda ) = (\lambda + \tfrac{1}{4} n)^{-1} (c - \tfrac{1}{2} n)[/math].

  • Find an expression for [math]\mathbb{E} [ \hat{\beta}_{1^{\mbox{{\tiny st}}}} (\lambda) ][/math].
  • Find an expression for [math]\mbox{Var} [ \hat{\beta}_{1^{\mbox{{\tiny st}}}} (\lambda) ][/math].
  • Combine the answers of parts d) and e) to find an expression for [math]\mbox{MSE} [ \hat{\beta}_{1^{\mbox{{\tiny st}}}} (\lambda) ][/math].
  • Plot this MSE against [math]\lambda[/math] for the data provided in Table and reflect on analogue of Theorem for the ridge logistic regression estimator.
  1. van Wieringen, W. and Aflakparast, M. (2021).porridge: Ridge-Type Estimation of a Potpourri of Models.R package version 0.2.1