Linear Models and Regression Questions and Answers

Answer: D

The estimable function is \(\alpha_{1} + \alpha_{p} - 2\alpha_{3}\).

In the one-way model, \(E(y_{ij}) = \mu + \alpha_{i}\), and a linear function \(c_{0}\mu + \sum c_{i}\alpha_{i}\) is estimable exactly when it is a combination of these expectations, that is, when \(c_{0} = \sum c_{i}\). With no \(\mu\) term, this means \(\sum c_{i} = 0\): a treatment contrast.

\(\alpha_{1} + \alpha_{p} - 2\alpha_{3}\) has coefficients summing to \(1 + 1 - 2 = 0\), so it is a contrast and estimable. \(\alpha_{1} + \alpha_{3}\) and \(2\alpha_{1} - \alpha_{2}\) have sums 2 and 1, and \(\mu + \alpha_{1} - \alpha_{2}\) has \(c_{0} = 1\) but \(\sum c_{i} = 0\), so none of these is estimable.

Answer: B

The correct statement is the BLUE of \(\theta_{2}\) is \(y_{2} - (y_{1} - y_{3})/2\) with variance \(\frac{3}{2}\sigma^{2}\).

Three observations and three parameters give an exactly determined system: \(y_{1} - y_{3}\) estimates \(2\theta_{3}\), \(y_{1} + y_{3}\) estimates \(2\theta_{1}\), and then \(y_{2} - \hat{\theta}_{3}\) estimates \(\theta_{2}\). Each estimable function has a unique linear unbiased estimator, which is therefore the BLUE.

\[ \begin{aligned} \mathrm{Var}\left(y_{2} - \tfrac{y_{1} - y_{3}}{2}\right) &= \sigma^{2} + \tfrac{1}{4}(2\sigma^{2}) \\ &= \tfrac{3}{2}\sigma^{2} \end{aligned} \]

\(y_{2}\) alone estimates \(\theta_{2} + \theta_{3}\), not \(\theta_{2}\). The BLUE of \(\theta_{1}\), \((y_{1} + y_{3})/2\), has variance \(\sigma^{2}/2\), not \(\sigma^{2}\), and \((y_{1} + y_{2})/2\) is biased for \(\theta_{1}\).

Answer: A

The fitted line will definitely pass through \((\bar{x}, \bar{y})\) when the intercept is present.

With an intercept, the first normal equation is \(\sum(y_{i} - a - bx_{i}) = 0\), which gives \(a = \bar{y} - b\bar{x}\). So \(\hat{y} = \bar{y} + b(x - \bar{x})\), and at \(x = \bar{x}\) the line gives \(\bar{y}\), whatever the sign of the slope.

Without an intercept (regression through the origin) there is no such normal equation, and the line generally misses \((\bar{x}, \bar{y})\), so (B) is too strong. 'Never' is wrong, and the slope's sign is irrelevant.

Answer: B

The responses are non-identical and uncorrelated.

\(E(y_{i}) = x_{i}'B\) depends on the \(i\)th row of X, so the responses have different means and are not identically distributed. The assumption \(\mathrm{Var}(E) = \sigma^{2}I\) gives them a common variance and zero covariances, so they are uncorrelated.

Independence is not implied: only first and second moments are assumed, and uncorrelated variables need not be independent unless normality is added. So 'non-identical and independent' says more than the model gives, and the 'identical' options ignore the different means.

Answer: C

The statement that is not correct is \(r_{12.3} \ge 0\) always.

A partial correlation is an ordinary correlation, between the residuals of \(X_{1}\) and \(X_{2}\) after each is regressed on \(X_{3}\) (A), so it lies between -1 and 1 (B) and can be negative. For example, with \(r_{12} = 0\), \(r_{13} = r_{23} = 0.5\), the formula gives

\[ \begin{aligned} r_{12.3} &= \frac{0 - 0.25}{\sqrt{0.75 \times 0.75}} \\ &= -\frac{1}{3} \end{aligned} \]

Option D is the standard formula for the partial correlation in terms of the simple correlations, so it is correct.

Answer: C

The key's answer is that A is an orthogonal matrix.

\(Y = AX\) is normal with covariance \(\sigma^{2}AA'\). Its components are uncorrelated when \(AA'\) is diagonal. An orthogonal A gives \(AA' = I\), so \(Y \sim N_{n}(A\mu, \sigma^{2}I)\): the components are uncorrelated (indeed independent) with the same variance, which is the property the question has in mind.

A general non-singular A gives \(AA'\) with non-zero off-diagonal entries. An idempotent matrix of rank below \(n\) makes \(\sigma^{2}AA'\) singular and correlated in general, and a matrix of all 1s makes every component equal to \(\sum X_{i}\), perfectly correlated.

Answer: B

The correct statement is that \(2\theta_{2} + 3\theta_{3}\) is not estimable.

A linear function is estimable when it is a combination of the expected values. The available expectations are \(\theta_{1} + 2\theta_{2} + \theta_{3}\) and \(\theta_{1} + \theta_{3}\), so every estimable function has the form

\[ (a + b)\theta_{1} + 2a\theta_{2} + (a + b)\theta_{3} \]

with equal coefficients of \(\theta_{1}\) and \(\theta_{3}\). \(2\theta_{2} + 3\theta_{3}\) has coefficients 0 and 3, so it is not estimable.

The others are estimable: \(\theta_{2}\) (\(a = 1/2, b = -1/2\)), \(\theta_{1} + \theta_{3}\) (\(a = 0, b = 1\)) and \(\theta_{1} + \theta_{2} + \theta_{3}\) (\(a = b = 1/2\)). So A, C and D are false.

Answer: D

Normality of the errors is needed for using the F test to test the model fit.

Least squares estimation of \(\underline{\theta}\), the Gauss-Markov theorem (BLUE property), the unbiased estimator \(\hat{\sigma}^{2} = \text{RSS}/(n - p)\), and the decomposition of the total sum of squares into regression and residual parts that are orthogonal all follow from \(E(\xi) = 0\) and \(\text{cov}(\xi) = \sigma^{2} I_{n}\) alone. They are geometry and second moments.

The exact sampling distributions are another matter. That the sums of squares divided by \(\sigma^{2}\) are independent chi-square variables, and hence that their ratio follows an F distribution, is Cochran's theorem for normal vectors. Without normality the F test is only approximate. So the first three options do not need the normality assumption, and the fourth does.

Answer: A

The p-value is much larger than 0.05.

The p-value is the probability, under the null hypothesis, of an F ratio at least as large as the one observed. Data that are fully consistent with equal treatment effects give a small F ratio, close to 1 or below, and such values are common under \(H_{0}\), so the p-value is large and the null is not rejected at any usual level.

A p-value between 0.01 and 0.05, or below 0.01, would be evidence against the null hypothesis, the opposite of the situation described. And something can be said: support for \(H_{0}\) translates directly into a large p-value. (Strictly, a large p-value means the data give no evidence against the null, rather than proving it.)

Answer: A

The least squares line may be used to predict a value of Y given the value of X.

That is the purpose of the fitted regression of Y on X: \(\hat{y} = \hat{\beta}_{0} + \hat{\beta}_{1}x\) gives the predicted response at a given \(x\), ideally within the range of the observed data.

The other statements are false. A regression line describes association, and does not by itself imply that X causes Y; that needs a designed experiment or further argument. The least squares line can be computed for any data set with some variation in X, whether or not the true relationship is linear; whether a straight line is a good description is a separate question answered by residual analysis. Since two of the three statements fail, 'all the above' is wrong.