A linear model has been fit to a dataset containing six predictor variables, F, G, H, I, J, and K. Determine which of the following statements regarding using Akaike information criterion (AIC) or Bayesian information criterion (BIC) to select an optimal set of predictor variables for this linear model is/are true.

• AIC and BIC provide a direct estimate of the test error.
• When choosing between the subsets {F, G, H} and {I, J, K}, AIC and BIC will always select the same subset.
• For large sample sizes (n > 7), the number of variables selected by BIC will be less than or equal to the number selected by AIC.

• None
• I and II only
• I and III only
• II and III only
• The correct answer is not given by (A), (B), (C), or (D).

Key: D

I is false, AIC and BIC make an indirect estimate by adjusting the training error.

II is true, for a fixed value of the number of predictors, the two provide the same ranking.

III is true as for n > 7, BIC provides a greater penalty for additional variables, and hence will select a number less than equal to that selected by AIC.

Copyright 2023. The Society of Actuaries, Schaumburg, Illinois. Reproduced with permission.