Linear Models and Regression Questions and Answers

Q: Under one way fixed effects ANOVA which of the following parametric functions are estimable, where the model is :\(y_{ij} = \mu + \alpha_{i} + \epsilon_{ij}\) \(i = 1 \ldots p,\ j = 1 \ldots n_{i},\)\(\epsilon_{ij}\) are independent identically distributed with zero mean and finite variance.

The estimable function is \(\alpha_{1} + \alpha_{p} - 2\alpha_{3}\).In the one-way model, \(E(y_{ij}) = \mu + \alpha_{i}\), and a linear function \(c_{0}\mu + \sum c_{i}\alpha_{i}\) is estimable exactly when it is a combination of these expectations, that is, when \(c_{0} = \sum c_{i}\). With no \(\mu\) term, this means \(\sum c_{i} = 0\): a treatment contrast.\(\alpha_{1} + \alpha_{p} - 2\alpha_{3}\) has coefficients summing to \(1 + 1 - 2 = 0\), so it is a contrast and estimable. \(\alpha_{1} + \alpha_{3}\) and \(2\alpha_{1} - \alpha_{2}\) have sums 2 and 1, and \(\mu + \alpha_{1} - \alpha_{2}\) has \(c_{0} = 1\) but \(\sum c_{i} = 0\), so none of these is estimable.

Q: \(y_{1}, y_{2}, y_{3}\) are three uncorrelated random variables with common variance \(\sigma^{2}\) and \(E(y_{1}) = \theta_{1} + \theta_{3}\), \(E(y_{2}) = \theta_{2} + \theta_{3}\), \(E(y_{3}) = \theta_{1} - \theta_{3}\). Hence :

The correct statement is the BLUE of \(\theta_{2}\) is \(y_{2} - (y_{1} - y_{3})/2\) with variance \(\frac{3}{2}\sigma^{2}\).Three observations and three parameters give an exactly determined system: \(y_{1} - y_{3}\) estimates \(2\theta_{3}\), \(y_{1} + y_{3}\) estimates \(2\theta_{1}\), and then \(y_{2} - \hat{\theta}_{3}\) estimates \(\theta_{2}\). Each estimable function has a unique linear unbiased estimator, which is therefore the BLUE.\[ \begin{aligned} \mathrm{Var}\left(y_{2} - \tfrac{y_{1} - y_{3}}{2}\right) &= \sigma^{2} + \tfrac{1}{4}(2\sigma^{2}) \\ &= \tfrac{3}{2}\sigma^{2} \end{aligned} \]\(y_{2}\) alone estimates \(\theta_{2} + \theta_{3}\), not \(\theta_{2}\). The BLUE of \(\theta_{1}\), \((y_{1} + y_{3})/2\), has variance \(\sigma^{2}/2\), not \(\sigma^{2}\), and \((y_{1} + y_{2})/2\) is biased for \(\theta_{1}\).

Q: In a simple linear regression analysis the fitted line using least squares theory will :

The fitted line will definitely pass through \((\bar{x}, \bar{y})\) when the intercept is present.With an intercept, the first normal equation is \(\sum(y_{i} - a - bx_{i}) = 0\), which gives \(a = \bar{y} - b\bar{x}\). So \(\hat{y} = \bar{y} + b(x - \bar{x})\), and at \(x = \bar{x}\) the line gives \(\bar{y}\), whatever the sign of the slope.Without an intercept (regression through the origin) there is no such normal equation, and the line generally misses \((\bar{x}, \bar{y})\), so (B) is too strong. 'Never' is wrong, and the slope's sign is irrelevant.

Q: In a multiple linear regression set up \(y = XB + E\) under the assumption \(E(E) = 0\), \(\mathrm{Var}(E) = \sigma^{2}\). In the response variables \(y_{1} \ldots y_{n}\) are :

The responses are non-identical and uncorrelated.\(E(y_{i}) = x_{i}'B\) depends on the \(i\)th row of X, so the responses have different means and are not identically distributed. The assumption \(\mathrm{Var}(E) = \sigma^{2}I\) gives them a common variance and zero covariances, so they are uncorrelated.Independence is not implied: only first and second moments are assumed, and uncorrelated variables need not be independent unless normality is added. So 'non-identical and independent' says more than the model gives, and the 'identical' options ignore the different means.

Q: Suppose \(r_{12.3}\) is a partial correlation coefficient between \((X_{1}, X_{2})\) and \(X_{3}\). Then which of the following statements is not correct ?

The statement that is not correct is \(r_{12.3} \ge 0\) always.A partial correlation is an ordinary correlation, between the residuals of \(X_{1}\) and \(X_{2}\) after each is regressed on \(X_{3}\) (A), so it lies between -1 and 1 (B) and can be negative. For example, with \(r_{12} = 0\), \(r_{13} = r_{23} = 0.5\), the formula gives\[ \begin{aligned} r_{12.3} &= \frac{0 - 0.25}{\sqrt{0.75 \times 0.75}} \\ &= -\frac{1}{3} \end{aligned} \]Option D is the standard formula for the partial correlation in terms of the simple correlations, so it is correct.

Q: Let \(\underline{X} \sim N_{n}(\mu, \sigma^{2}I)\) and A be a \(n \times n\) matrix. Then \(\underline{Y} = AX\) are uncorrelated normally distributed if and only if which of the following conditions hold ?

The key's answer is that A is an orthogonal matrix.\(Y = AX\) is normal with covariance \(\sigma^{2}AA'\). Its components are uncorrelated when \(AA'\) is diagonal. An orthogonal A gives \(AA' = I\), so \(Y \sim N_{n}(A\mu, \sigma^{2}I)\): the components are uncorrelated (indeed independent) with the same variance, which is the property the question has in mind.A general non-singular A gives \(AA'\) with non-zero off-diagonal entries. An idempotent matrix of rank below \(n\) makes \(\sigma^{2}AA'\) singular and correlated in general, and a matrix of all 1s makes every component equal to \(\sum X_{i}\), perfectly correlated.

Q: \(y_{1}, y_{2}, y_{3}, y_{4}\) are four uncorrelated random variables with common variance \(\sigma^{2}\) and \(E(y_{1}) = E(y_{2}) = \theta_{1} + 2\theta_{2} + \theta_{3}\), \(E(y_{3}) = E(y_{4}) = \theta_{1} + \theta_{3}\). Hence :

The correct statement is that \(2\theta_{2} + 3\theta_{3}\) is not estimable.A linear function is estimable when it is a combination of the expected values. The available expectations are \(\theta_{1} + 2\theta_{2} + \theta_{3}\) and \(\theta_{1} + \theta_{3}\), so every estimable function has the form\[ (a + b)\theta_{1} + 2a\theta_{2} + (a + b)\theta_{3} \]with equal coefficients of \(\theta_{1}\) and \(\theta_{3}\). \(2\theta_{2} + 3\theta_{3}\) has coefficients 0 and 3, so it is not estimable.The others are estimable: \(\theta_{2}\) (\(a = 1/2, b = -1/2\)), \(\theta_{1} + \theta_{3}\) (\(a = 0, b = 1\)) and \(\theta_{1} + \theta_{2} + \theta_{3}\) (\(a = b = 1/2\)). So A, C and D are false.

Q: Consider the linear model \(\underline{y}_{n \times 1} = X\underline{\theta}_{p \times 1} + \xi_{n \times 1}\) where \(E(\xi) = 0\), \(\text{cov}(\xi) = \sigma^{2} I_{n}\). The assumption that \(\xi\) is multivariate Normal is needed for :

Normality of the errors is needed for using the F test to test the model fit.Least squares estimation of \(\underline{\theta}\), the Gauss-Markov theorem (BLUE property), the unbiased estimator \(\hat{\sigma}^{2} = \text{RSS}/(n - p)\), and the decomposition of the total sum of squares into regression and residual parts that are orthogonal all follow from \(E(\xi) = 0\) and \(\text{cov}(\xi) = \sigma^{2} I_{n}\) alone. They are geometry and second moments.The exact sampling distributions are another matter. That the sums of squares divided by \(\sigma^{2}\) are independent chi-square variables, and hence that their ratio follows an F distribution, is Cochran's theorem for normal vectors. Without normality the F test is only approximate. So the first three options do not need the normality assumption, and the fourth does.

Q: In case of a one-way ANOVA, the data strongly supports the null hypothesis that all the treatments have the same effect. Hence the p-value corresponding to the F ratio is :

The p-value is much larger than 0.05.The p-value is the probability, under the null hypothesis, of an F ratio at least as large as the one observed. Data that are fully consistent with equal treatment effects give a small F ratio, close to 1 or below, and such values are common under \(H_{0}\), so the p-value is large and the null is not rejected at any usual level.A p-value between 0.01 and 0.05, or below 0.01, would be evidence against the null hypothesis, the opposite of the situation described. And something can be said: support for \(H_{0}\) translates directly into a large p-value. (Strictly, a large p-value means the data give no evidence against the null, rather than proving it.)

Q: In simple linear regression with regression X and response variate Y the least squares regression line :

The least squares line may be used to predict a value of Y given the value of X.That is the purpose of the fitted regression of Y on X: \(\hat{y} = \hat{\beta}_{0} + \hat{\beta}_{1}x\) gives the predicted response at a given \(x\), ideally within the range of the observed data.The other statements are false. A regression line describes association, and does not by itself imply that X causes Y; that needs a designed experiment or further argument. The least squares line can be computed for any data set with some variation in X, whether or not the true relationship is linear; whether a straight line is a good description is a separate question answered by residual analysis. Since two of the three statements fail, 'all the above' is wrong.

11.

Under one way fixed effects ANOVA which of the following parametric functions are estimable, where the model is :

\(y_{ij} = \mu + \alpha_{i} + \epsilon_{ij}\) \(i = 1 \ldots p,\ j = 1 \ldots n_{i},\)

\(\epsilon_{ij}\) are independent identically distributed with zero mean and finite variance.

A.\(\mu + \alpha_{1} - \alpha_{2}\)
B.\(\alpha_{1} + \alpha_{3}\)
C.\(2\alpha_{1} - \alpha_{2}\)
D.\(\alpha_{1} + \alpha_{p} - 2\alpha_{3}\)

Exam Year(s)

MH SET - 2020, SET Exam - 2020,

Answer: D

The estimable function is \(\alpha_{1} + \alpha_{p} - 2\alpha_{3}\).

In the one-way model, \(E(y_{ij}) = \mu + \alpha_{i}\), and a linear function \(c_{0}\mu + \sum c_{i}\alpha_{i}\) is estimable exactly when it is a combination of these expectations, that is, when \(c_{0} = \sum c_{i}\). With no \(\mu\) term, this means \(\sum c_{i} = 0\): a treatment contrast.

\(\alpha_{1} + \alpha_{p} - 2\alpha_{3}\) has coefficients summing to \(1 + 1 - 2 = 0\), so it is a contrast and estimable. \(\alpha_{1} + \alpha_{3}\) and \(2\alpha_{1} - \alpha_{2}\) have sums 2 and 1, and \(\mu + \alpha_{1} - \alpha_{2}\) has \(c_{0} = 1\) but \(\sum c_{i} = 0\), so none of these is estimable.

Discuss in Forum

12.

\(y_{1}, y_{2}, y_{3}\) are three uncorrelated random variables with common variance \(\sigma^{2}\) and \(E(y_{1}) = \theta_{1} + \theta_{3}\), \(E(y_{2}) = \theta_{2} + \theta_{3}\), \(E(y_{3}) = \theta_{1} - \theta_{3}\). Hence :

A.Best Linear Unbiased Estimator (BLUE) of \(\theta_{2}\) is \(y_{2}\) with variance \(\sigma^{2}\)
B.BLUE of \(\theta_{2}\) is \(y_{2} - \dfrac{(y_{1} - y_{3})}{2}\) with variance \(\dfrac{3}{2}\sigma^{2}\)
C.BLUE of \(\theta_{1}\) is \(\dfrac{y_{1} + y_{3}}{2}\) with variance \(\sigma^{2}\)
D.BLUE of \(\theta_{1}\) is \(\dfrac{y_{1} + y_{2}}{2}\) with variance \(\dfrac{3\sigma^{2}}{2}\)

Exam Year(s)

MH SET - 2020, SET Exam - 2020,

Answer: B

The correct statement is the BLUE of \(\theta_{2}\) is \(y_{2} - (y_{1} - y_{3})/2\) with variance \(\frac{3}{2}\sigma^{2}\).

Three observations and three parameters give an exactly determined system: \(y_{1} - y_{3}\) estimates \(2\theta_{3}\), \(y_{1} + y_{3}\) estimates \(2\theta_{1}\), and then \(y_{2} - \hat{\theta}_{3}\) estimates \(\theta_{2}\). Each estimable function has a unique linear unbiased estimator, which is therefore the BLUE.

\[ \begin{aligned} \mathrm{Var}\left(y_{2} - \tfrac{y_{1} - y_{3}}{2}\right) &= \sigma^{2} + \tfrac{1}{4}(2\sigma^{2}) \\ &= \tfrac{3}{2}\sigma^{2} \end{aligned} \]

\(y_{2}\) alone estimates \(\theta_{2} + \theta_{3}\), not \(\theta_{2}\). The BLUE of \(\theta_{1}\), \((y_{1} + y_{3})/2\), has variance \(\sigma^{2}/2\), not \(\sigma^{2}\), and \((y_{1} + y_{2})/2\) is biased for \(\theta_{1}\).

Discuss in Forum

13.

In a simple linear regression analysis the fitted line using least squares theory will :

A.definitely pass through the point \((\bar{x}\ \bar{y})\) when the intercept is present
B.definitely pass through the point \((\bar{x}\ \bar{y})\)
C.never pass through the point \((\bar{x}\ \bar{y})\)
D.definitely pass through the point \((\bar{x}\ \bar{y})\) if slope is positive

Exam Year(s)

MH SET - 2020, SET Exam - 2020,

Answer: A

The fitted line will definitely pass through \((\bar{x}, \bar{y})\) when the intercept is present.

With an intercept, the first normal equation is \(\sum(y_{i} - a - bx_{i}) = 0\), which gives \(a = \bar{y} - b\bar{x}\). So \(\hat{y} = \bar{y} + b(x - \bar{x})\), and at \(x = \bar{x}\) the line gives \(\bar{y}\), whatever the sign of the slope.

Without an intercept (regression through the origin) there is no such normal equation, and the line generally misses \((\bar{x}, \bar{y})\), so (B) is too strong. 'Never' is wrong, and the slope's sign is irrelevant.

Discuss in Forum

14.

In a multiple linear regression set up \(y = XB + E\) under the assumption \(E(E) = 0\), \(\mathrm{Var}(E) = \sigma^{2}\). In the response variables \(y_{1} \ldots y_{n}\) are :

A.identical and independent
B.non-identical uncorrelated
C.non-identical and independent
D.identical and uncorrelated

Exam Year(s)

MH SET - 2020, SET Exam - 2020,

Answer: B

The responses are non-identical and uncorrelated.

\(E(y_{i}) = x_{i}'B\) depends on the \(i\)th row of X, so the responses have different means and are not identically distributed. The assumption \(\mathrm{Var}(E) = \sigma^{2}I\) gives them a common variance and zero covariances, so they are uncorrelated.

Independence is not implied: only first and second moments are assumed, and uncorrelated variables need not be independent unless normality is added. So 'non-identical and independent' says more than the model gives, and the 'identical' options ignore the different means.

Discuss in Forum

15.

Suppose \(r_{12.3}\) is a partial correlation coefficient between \((X_{1}, X_{2})\) and \(X_{3}\). Then which of the following statements is not correct ?

A.\(r_{12.3}\) is a correlation coefficient between \(X_{1} - \hat{X}_{1}\) and \(X_{2} - \hat{X}_{2}\), where \(\hat{X}_{1}\) is a line of best fit based on \(X_{3}\) and \(\hat{X}_{2}\) is a line of best fit based on \(X_{3}\).
B.\(-1 \le r_{12.3} \le 1\)
C.\(r_{12.3} \ge 0\) always
D.\(r_{12.3} =\) \(\frac{r_{12} - r_{13}.r_{23}}{\sqrt{(1 - r_{13}^{2})(1 - r_{23}^{2})}}\)

Exam Year(s)

MH SET - 2021, SET Exam - 2021,

Answer: C

The statement that is not correct is \(r_{12.3} \ge 0\) always.

A partial correlation is an ordinary correlation, between the residuals of \(X_{1}\) and \(X_{2}\) after each is regressed on \(X_{3}\) (A), so it lies between -1 and 1 (B) and can be negative. For example, with \(r_{12} = 0\), \(r_{13} = r_{23} = 0.5\), the formula gives

\[ \begin{aligned} r_{12.3} &= \frac{0 - 0.25}{\sqrt{0.75 \times 0.75}} \\ &= -\frac{1}{3} \end{aligned} \]

Option D is the standard formula for the partial correlation in terms of the simple correlations, so it is correct.

Discuss in Forum

16.

Let \(\underline{X} \sim N_{n}(\mu, \sigma^{2}I)\) and A be a \(n \times n\) matrix. Then \(\underline{Y} = AX\) are uncorrelated normally distributed if and only if which of the following conditions hold ?

A.A is any non-singular matrix
B.A is an idempotent matrix with rank \(\lt n\)
C.A is an orthogonal matrix
D.All elements of A are equal to 1

Exam Year(s)

MH SET - 2021, SET Exam - 2021,

Answer: C

The key's answer is that A is an orthogonal matrix.

\(Y = AX\) is normal with covariance \(\sigma^{2}AA'\). Its components are uncorrelated when \(AA'\) is diagonal. An orthogonal A gives \(AA' = I\), so \(Y \sim N_{n}(A\mu, \sigma^{2}I)\): the components are uncorrelated (indeed independent) with the same variance, which is the property the question has in mind.

A general non-singular A gives \(AA'\) with non-zero off-diagonal entries. An idempotent matrix of rank below \(n\) makes \(\sigma^{2}AA'\) singular and correlated in general, and a matrix of all 1s makes every component equal to \(\sum X_{i}\), perfectly correlated.

Discuss in Forum

17.

\(y_{1}, y_{2}, y_{3}, y_{4}\) are four uncorrelated random variables with common variance \(\sigma^{2}\) and \(E(y_{1}) = E(y_{2}) = \theta_{1} + 2\theta_{2} + \theta_{3}\), \(E(y_{3}) = E(y_{4}) = \theta_{1} + \theta_{3}\). Hence :

A.\(\theta_{2}\) is not estimable
B.\(2\theta_{2} + 3\theta_{3}\) is not estimable
C.\(\theta_{1} + \theta_{3}\) is not estimable
D.\(\theta_{1} + \theta_{2} + \theta_{3}\) is not estimable

Exam Year(s)

MH SET - 2021, SET Exam - 2021,

Answer: B

The correct statement is that \(2\theta_{2} + 3\theta_{3}\) is not estimable.

A linear function is estimable when it is a combination of the expected values. The available expectations are \(\theta_{1} + 2\theta_{2} + \theta_{3}\) and \(\theta_{1} + \theta_{3}\), so every estimable function has the form

\[ (a + b)\theta_{1} + 2a\theta_{2} + (a + b)\theta_{3} \]

with equal coefficients of \(\theta_{1}\) and \(\theta_{3}\). \(2\theta_{2} + 3\theta_{3}\) has coefficients 0 and 3, so it is not estimable.

The others are estimable: \(\theta_{2}\) (\(a = 1/2, b = -1/2\)), \(\theta_{1} + \theta_{3}\) (\(a = 0, b = 1\)) and \(\theta_{1} + \theta_{2} + \theta_{3}\) (\(a = b = 1/2\)). So A, C and D are false.

Discuss in Forum

18.

Consider the linear model \(\underline{y}_{n \times 1} = X\underline{\theta}_{p \times 1} + \xi_{n \times 1}\) where \(E(\xi) = 0\), \(\text{cov}(\xi) = \sigma^{2} I_{n}\). The assumption that \(\xi\) is multivariate Normal is needed for :

A.estimating the parameter \(\underline{\theta}\)
B.estimating the parameter \(\sigma^{2}\)
C.splitting the total sum of squares into two orthogonal parts
D.using F test to test the model fit

Exam Year(s)

MH SET - 2023, SET Exam - 2023,

Answer: D

Normality of the errors is needed for using the F test to test the model fit.

Least squares estimation of \(\underline{\theta}\), the Gauss-Markov theorem (BLUE property), the unbiased estimator \(\hat{\sigma}^{2} = \text{RSS}/(n - p)\), and the decomposition of the total sum of squares into regression and residual parts that are orthogonal all follow from \(E(\xi) = 0\) and \(\text{cov}(\xi) = \sigma^{2} I_{n}\) alone. They are geometry and second moments.

The exact sampling distributions are another matter. That the sums of squares divided by \(\sigma^{2}\) are independent chi-square variables, and hence that their ratio follows an F distribution, is Cochran's theorem for normal vectors. Without normality the F test is only approximate. So the first three options do not need the normality assumption, and the fourth does.

Discuss in Forum

19.

In case of a one-way ANOVA, the data strongly supports the null hypothesis that all the treatments have the same effect. Hence the p-value corresponding to the F ratio is :

A.much larger than 0.05
B.between 0.01 and 0.05
C.much smaller than 0.01
D.nothing can be said about the p-value

Exam Year(s)

MH SET - 2023, SET Exam - 2023,

Answer: A

The p-value is much larger than 0.05.

The p-value is the probability, under the null hypothesis, of an F ratio at least as large as the one observed. Data that are fully consistent with equal treatment effects give a small F ratio, close to 1 or below, and such values are common under \(H_{0}\), so the p-value is large and the null is not rejected at any usual level.

A p-value between 0.01 and 0.05, or below 0.01, would be evidence against the null hypothesis, the opposite of the situation described. And something can be said: support for \(H_{0}\) translates directly into a large p-value. (Strictly, a large p-value means the data give no evidence against the null, rather than proving it.)

Discuss in Forum

20.

In simple linear regression with regression X and response variate Y the least squares regression line :

A.may be used to predict a value of Y given the value of X
B.implies a cause-effect relationship between X and Y
C.can be determined only if a linear relationship exists between Y and X
D.all the above statements are true

Exam Year(s)

MH SET - 2023, SET Exam - 2023,

Answer: A

The least squares line may be used to predict a value of Y given the value of X.

That is the purpose of the fitted regression of Y on X: \(\hat{y} = \hat{\beta}_{0} + \hat{\beta}_{1}x\) gives the predicted response at a given \(x\), ideally within the range of the observed data.

The other statements are false. A regression line describes association, and does not by itself imply that X causes Y; that needs a designed experiment or further argument. The least squares line can be computed for any data set with some variation in X, whether or not the true relationship is linear; whether a straight line is a good description is a separate question answered by residual analysis. Since two of the three statements fail, 'all the above' is wrong.

Linear Models and Regression Questions and Answers

Topics

Other Subjects

Tags: Linear Models and Regression Questions and Answers

About Us

Linear Models and Regression Questions and Answers

Topics

Other Subjects

Tags: Linear Models and Regression Questions and Answers

About Us

Stay informed about all important Dates & Notifications