Linear Models and Regression Questions and Answers

Answer: C

The correct statement is that the mean error ss is unbiased for \(\sigma^{2}\), while the mean regression ss and mean total ss are biased.

The paper prints the model as \(\underline{Y} = r\underline{B} + \underline{\varepsilon}\) with covariance \(\sigma^{2} t_{n}\); this is the usual linear model \(Y = X\beta + \varepsilon\) with \(\text{cov}(\varepsilon) = \sigma^{2}I_{n}\). In that model

\[ \begin{aligned} E(\text{MSE}) &= \sigma^{2} \\ E(\text{MSR}) &= \sigma^{2} + \frac{\beta' X' (I - J/n) X \beta}{p - 1} \end{aligned} \]

The residual mean square is always unbiased. The regression mean square exceeds \(\sigma^{2}\) by a non-negative term that is zero only when all slope coefficients are zero, which is why their ratio tests that hypothesis. The total mean square mixes the two and is biased for the same reason. So 'all three unbiased', 'regression and error unbiased' and 'all three biased' are wrong.

Answer: C

The answer is option C: the equality holds if the regression equation has an intercept.

When the model is \(y = \beta_{0} + \beta_{1}X + \varepsilon\), the first normal equation gives \(\sum (y_{i} - \hat{y}_{i}) = 0\), so \(\sum y_{i} = \sum \hat{y}_{i}\). Option C states this as a sufficient condition. A is false, because a model through the origin need not give equal sums. B asks for more than is needed (only if), and D (if and only if) is also too strong: the equality holds for other models with an intercept and, for particular data, can hold without one, so neither is a true statement of the rule.

Answer: A

In the logistic regression model the response is binary.

Logistic regression models the probability of a two-category outcome, success or failure, coded 1 and 0, through

\[ \begin{aligned} \log\frac{p}{1 - p} &= \beta_{0} + \beta_{1}x_{1} \\ &\quad + \cdots + \beta_{k}x_{k} \end{aligned} \]

so each response is a Bernoulli variable with success probability \(p\) depending on the predictors.

A response with finitely many (more than two) categories needs multinomial or ordinal logistic regression. Counts with a countably infinite range are handled by Poisson regression, and a continuous response by ordinary linear regression. These are the generalisations the distractors describe, not the logistic model itself.

Answer: C

The BLUE of \(\beta_{2}\) is \(\dfrac{Y_{1} - 5Y_{2} + 4Y_{3}}{14}\).

The design matrix has rows (1, 1), (1, -1), (1, 2). By the Gauss-Markov theorem the BLUE is the least squares estimator \((X'X)^{-1}X'Y\):

\[ \begin{aligned} X'X &= \begin{pmatrix} 3 & 2 \\ 2 & 6 \end{pmatrix},\quad (X'X)^{-1} = \frac{1}{14}\begin{pmatrix} 6 & -2 \\ -2 & 3 \end{pmatrix} \\ X'Y &= (Y_{1} + Y_{2} + Y_{3},\ Y_{1} - Y_{2} + 2Y_{3})' \end{aligned} \]

\[ \begin{aligned} \hat{\beta}_{2} &= \frac{-2(Y_{1} + Y_{2} + Y_{3}) + 3(Y_{1} - Y_{2} + 2Y_{3})}{14} \\ &= \frac{Y_{1} - 5Y_{2} + 4Y_{3}}{14} \end{aligned} \]

Check unbiasedness: the coefficients give \(\beta_{1}(1 - 5 + 4)/14 + \beta_{2}(1 + 5 + 8)/14 = \beta_{2}\). The first option has expectation \(\beta_{1} + 2\beta_{2}/3\), the second has expectation \(\beta_{1}\), and the last has expectation \(\frac{4 - 8 + 4}{3}\beta_{1} + \frac{4 + 8 + 8}{3}\beta_{2}\), which is \(\frac{20}{3}\beta_{2}\).

Answer: C

The key gives both (1) and (2).

(1) The regression coefficients are \(\beta_{YX} = r\frac{\sigma_{Y}}{\sigma_{X}}\) and \(\beta_{XY} = r\frac{\sigma_{X}}{\sigma_{Y}}\), so \(\beta_{YX}\beta_{XY} = r^{2}\). The correlation coefficient is their geometric mean, \(r = \pm\sqrt{\beta_{YX}\beta_{XY}}\), taking the common sign of the two coefficients.

(2) is the standard textbook property that follows from the AM-GM inequality: for positive coefficients,

\[ \begin{aligned} \frac{\beta_{YX} + \beta_{XY}}{2} &\ge \sqrt{\beta_{YX}\beta_{XY}} \\ &= r \end{aligned} \]

Textbooks state it for the usual case of positive correlation; for negative correlation it holds for the magnitudes. On that reading both statements are true, so the options naming only one, or neither, are wrong.

Answer: D

The always-true statement is that l'θ is estimable only for l in the column space of X'.

A linear function \(\underline{l}'\underline{\theta}\) is estimable if some linear function \(\underline{a}'\underline{Y}\) has expectation \(\underline{l}'\underline{\theta}\) for all \(\theta\). Since \(E(\underline{a}'\underline{Y}) = \underline{a}'X\underline{\theta}\), this needs \(\underline{l} = X'\underline{a}\), that is, \(\underline{l}\) must lie in the column space of X' (the row space of X).

If X does not have full column rank, \(\underline{\theta}\) itself is not estimable, it has no BLUE, and some \(\underline{l}'\underline{\theta}\) are not estimable. So the other three statements hold only in the full rank case, not always.

Answer: B

The statement not true is all three parameters Q₁, Q₂, Q₃ are estimable.

The expectations are \(Q_{1} - Q_{2} + Q_{3}\), \(Q_{1}\) and \(Q_{3} - Q_{2}\). The first is the sum of the other two, so only two independent linear functions are estimable, spanned by \(Q_{1}\) and \(Q_{3} - Q_{2}\). With rank 2 and three parameters, they cannot all be estimable.

The other statements are true. \(Q_{2} - Q_{3} = -E(Y_{3})\) is estimable. \(Q_{1}\) is estimable, but \(Q_{2}\) and \(Q_{3}\) separately are not, since they appear only as \(Q_{3} - Q_{2}\). And \(2Q_{1} - Q_{2} + Q_{3} = E(Y_{1}) + E(Y_{2})\) is estimable.

Answer: D

The values are a = 25, b = 0.90 and c = 0.014.

Degrees of freedom and sums of squares add up to the totals, and the mean square is the sum of squares divided by its degrees of freedom:

\[ \begin{aligned} a &= 29 - 4 = 25 \\ b &= 1.25 - 0.35 = 0.90 \\ c &= \frac{0.35}{25} = 0.014 \end{aligned} \]

The options with a = 15 subtract wrongly, b = 0.35 copies the error sum of squares, and c = 0.015 or 0.14 misplace the decimal in \(0.35/25\).

Answer: D

The true statement is that the differences α_i − α_k are estimable.

The model is the one-way model \(y_{ij} = \mu + \alpha_{i} + e_{ij}\) (the paper prints the effect as \(d_{i}\) in the model and \(\alpha_{i}\) in the options). Here \(E(y_{ij}) = \mu + \alpha_{i}\), so the estimable functions are combinations of \(\mu + \alpha_{1}, \ldots, \mu + \alpha_{4}\). The difference \((\mu + \alpha_{i}) - (\mu + \alpha_{k}) = \alpha_{i} - \alpha_{k}\) is estimable, and so is every contrast \(\sum c_{i}\alpha_{i}\) with \(\sum c_{i} = 0\).

The design matrix has rank 4 for 5 parameters, so \(\mu\) and the individual \(\alpha_{i}\) are not estimable, and neither is an arbitrary linear function of the \(\alpha_{i}\), such as \(\alpha_{1}\) alone. (The paper's 'i = k' is meant as i ≠ k; with i = k the difference is 0 and trivially estimable.)

Answer: D

The residuals are correlated and heteroscedastic.

With hat matrix \(H = X(X'X)^{-1}X'\), the residual vector is \(e = (I - H)y = (I - H)\varepsilon\), so

\[ Cov(e) = \sigma^{2}(I - H) \]

The off-diagonal entries \(-\sigma^{2}h_{ij}\) are generally non-zero, so the residuals are correlated, and the diagonal entries \(\sigma^{2}(1 - h_{ii})\) vary with the leverages \(h_{ii}\), so they are heteroscedastic, even though the errors themselves are uncorrelated and homoscedastic.

This is why standardised and studentised residuals divide each \(e_{i}\) by \(\sqrt{1 - h_{ii}}\).