Linear Models and Regression Questions and Answers

Q: Which of the following statements is correct about a regression model ?

The keyed statement is residual sum of squares reduces with every new term added in the model.Least squares chooses the coefficients that minimise the residual sum of squares. A model with one more term can always reproduce the smaller model by setting the new coefficient to 0, so its minimum cannot be larger. Adding a term therefore never increases RSS; it falls whenever the new term explains any part of the residual, and stays the same only in the degenerate case where the new term is exactly uncorrelated with the current residuals.That is why RSS (and \(R^{2}\)) cannot be used alone to choose a model, and adjusted \(R^{2}\) or an F-test is needed. The statements saying RSS increases are wrong in every case, and the condition 'provided the response is dependent on the new term' is unnecessary: even an irrelevant term lowers RSS by chance.

Q: Let \(X_{1}\), \(X_{2}\), .......... \(X_{n}\) be iid r.v.s satisfying the following regression equation :\(X_{i} = \alpha z_{i} + e_{i}\); \(i = 1, 2, ....... n\)where \(z_{1}\), \(z_{2}\), ....... \(z_{n}\) are fixed and \(e_{i}\)'s \((i = 1, 2, ..... n)\) are iid r.v.s with \(N(0, \sigma^{2})\), \(\sigma^{2}\) is unknown MLE of \(\alpha\) is given as :

The MLE is \(\hat{\alpha} = \sum x_{i}z_{i}/\sum z_{i}^{2}\).With normal errors, maximising the likelihood in \(\alpha\) is the same as minimising \(\sum(x_{i} - \alpha z_{i})^{2}\), whatever \(\sigma^{2}\) is. Setting the derivative to zero:\[ \begin{aligned} -2\sum z_{i}(x_{i} - \alpha z_{i}) &= 0 \\ \hat{\alpha} &= \frac{\sum x_{i}z_{i}}{\sum z_{i}^{2}} \end{aligned} \]This is least squares through the origin. Option A divides by \(\sum x_{i}^{2}\), which would regress \(z\) on \(x\); option B divides by \(n\), which is correct only if every \(z_{i}^{2} = 1\); and option C does not involve the data at all.

Q: For a standard multiple regression model \(\{Y, X\beta, \sigma^{2}I\}\), which of the following statements is not true ?

The statement that is not true is \(\hat{Y} \sim N_{n}(\beta, \sigma^{2}H)\).With \(H = X(X'X)^{-1}X'\) the hat matrix, \(\hat{Y} = HY\). Since \(HX = X\), \(E(\hat{Y}) = HX\beta = X\beta\) (A). Since H is symmetric and idempotent, \(Cov(\hat{Y}) = H\sigma^{2}IH' = \sigma^{2}H\) (B). And \(Cov(HY, (I - H)Y) = \sigma^{2}H(I - H) = 0\) (C).Under normality \(\hat{Y}\) is normal with mean \(X\beta\), not \(\beta\). The vector \(\beta\) has \(p\) components while \(\hat{Y}\) has \(n\), so the stated mean is not even of the right size.

Q: Consider the multiple regression set-up \(\{Y, X\beta, \sigma^{2}I\}\) with \(p\)-regressors. The ordinary least squares estimator (OLSE) \(\hat{\beta}\) is the vector which minimizes \(E(Y - X\beta)(Y - X\beta)'\). Which of the following statements is not true under this set-up ?

The statement that is not true is \(\hat{\beta}\) is unique if and only if rank(X) \(\lt p\).The normal equations \(X'X\hat{\beta} = X'Y\) always have a solution, and the fitted vector \(X\hat{\beta}\) is always unique, being the orthogonal projection of Y onto the column space of X (A). The solution \(\hat{\beta}\) is unique exactly when \(X'X\) is invertible, that is when rank(X) = \(p\) (B). When rank(X) \(\lt p\) there are infinitely many solutions, which is the opposite of C.Even then, a linear function \(K'\hat{\beta}\) is the same for every solution precisely when \(K'\beta\) is estimable, which is D.

Q: Which of the following statements is not true in the context of the transformation of a response variable under a regression set-up ?

The statement that is not true is The Box-Cox transformation is a discontinuous function of \(\lambda\).The Box-Cox transformation is \(g(Y, \lambda) = (Y^{\lambda} - 1)/\lambda\) for \(\lambda \ne 0\) and \(\log Y\) for \(\lambda = 0\). The \(-1\) and the division by \(\lambda\) are there precisely to make it continuous in \(\lambda\): as \(\lambda \to 0\), \((Y^{\lambda} - 1)/\lambda \to \log Y\).The other statements are the standard variance-stabilising rules. When the variance equals the mean (Poisson-like data), \(\sqrt{Y}\) stabilises it (A). When the standard deviation is proportional to the mean, \(\log Y\) does (B). And a positive change of scale does not change the sign of the association (C).

Q: \(Y_{1}, Y_{2}\) and \(Y_{3}\) are three uncorrelated random variables with common variance \(\sigma^{2}\). Further \(E(Y_{1}) = \theta_{2} - \theta_{1}\), \(E(Y_{2}) = E(Y_{3}) = \theta_{2} + \theta_{3}\). Then :

The true statement is \(\theta_{1} + \theta_{2}\) is not estimable.A linear function is estimable when it is a linear combination of the expectations \(E(Y_{1}) = \theta_{2} - \theta_{1}\) and \(E(Y_{2}) = E(Y_{3}) = \theta_{2} + \theta_{3}\). For \(a(\theta_{2} - \theta_{1}) + b(\theta_{2} + \theta_{3})\) to equal \(\theta_{1} + \theta_{2}\), the \(\theta_{3}\) coefficient forces \(b = 0\), and then \(-a = 1\), \(a = 1\) contradict each other. So \(\theta_{1} + \theta_{2}\) is not estimable.The others are estimable, so their 'not estimable' claims are false: \(\theta_{1} - \theta_{2} = -E(Y_{1})\); \(\theta_{2} + \theta_{3} = E(Y_{2})\); and \(\theta_{1} + \theta_{3} = E(Y_{2}) - E(Y_{1})\).

Q: In a simple linear regression of Y on X based on \(n\) observations. The fitted values of \(Y_{i}\)'s are \(Y_{i}^{n}\), \(i = 1 ..... n\) respectively. Then :\[ \sum_{i=1}^{n} Y_{i} = \sum_{i=1}^{n} Y_{i}^{n} \]

The equality holds if the regression equation is \(Y = B_{0} + B_{1}X + \varepsilon\).The paper writes the fitted values as \(Y_{i}^{n}\). When the model has an intercept, the least squares normal equation for \(B_{0}\) is \(\sum(Y_{i} - \hat{Y}_{i}) = 0\): the residuals sum to zero, so \(\sum Y_{i} = \sum \hat{Y}_{i}\).It does not hold 'always': for regression through the origin, \(Y = B_{1}X + \varepsilon\), the residuals need not sum to zero. And 'only if' is too strong, because any model with an intercept (for example a multiple regression) has the same property. So the condition is sufficient, not necessary, which is what option D says.

Q: A least squares regression fit :

A least squares fit may be used to predict Y from given values of the regressor X.That is its purpose: the fitted equation \(\hat{Y} = b_{0} + b_{1}X\) gives the predicted response for a given X (sensibly within the range of the data).It does not show cause and effect: a regression measures association, and a strong fit can arise from a common cause or coincidence. Nor does fitting require a satisfactory relationship to exist: least squares produces a line for any data, however poor the fit. So B and C are false, and therefore D is too.

Q: Suppose \(Y_{i} = B_{0} + B_{1}(x_{i} - 2) + \varepsilon_{i}\), \(E(\varepsilon_{i}) = 0\), \(\mathrm{Var}(\varepsilon_{i}) = \sigma^{2}\) and where \(x_{i} = i\), \(i = 1, 2, 3\). The Best Linear Unbiased estimators of \(B_{0}\) and \(B_{1}\).

The BLUEs exist and are uncorrelated with each other.The regressor values are \(x_{i} - 2 = -1, 0, 1\), which sum to 0. The design matrix has full column rank, so the least squares estimators are the BLUEs (Gauss-Markov), and with a centred regressor\[ \hat{B}_0 = \bar{Y},\ \ \hat{B}_1 = \frac{Y_3 - Y_1}{2} \]Their covariance is \(-\sigma^{2}\bar{c}/S_{cc}\), where \(\bar{c} = 0\) is the mean of the centred regressor, so it is 0. Directly: \(\mathrm{Cov}(\bar{Y}, (Y_{3} - Y_{1})/2)\) is \(\tfrac{1}{6}(\sigma^{2} - \sigma^{2}) = 0\).Independence would need normal errors, which are not assumed, so C goes too far; and they are not positively correlated.

Q: The regression equation of Y on X is given by \(y = 2 - 1.2x\) and that of X on Y is given by \(x = -1 - 0.3y\). Hence the correlation coefficient between X and Y is :

The correlation coefficient is -0.6.The regression coefficients are \(b_{yx} = -1.2\) and \(b_{xy} = -0.3\), and \(r^{2} = b_{yx}b_{xy}\):\[ r^{2} = (-1.2)(-0.3) = 0.36, \quad |r| = 0.6 \]\(r\) has the same sign as the two regression coefficients, which are both negative, so \(r = -0.6\).0.36 is \(r^{2}\), not \(r\), and +0.6 has the wrong sign.

Which of the following statements is correct about a regression model ?

A.Residual sum of squares reduces with every new term added in the model.
B.Residual sum of squares reduces with new term added in the model provided the response variable is dependent on the new term.
C.Residual sum of squares increases with the new term added in the model.
D.Residual sum of squares increases with the new term added in the model provided the response variable is correlated with the new term.

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: A

The keyed statement is residual sum of squares reduces with every new term added in the model.

Least squares chooses the coefficients that minimise the residual sum of squares. A model with one more term can always reproduce the smaller model by setting the new coefficient to 0, so its minimum cannot be larger. Adding a term therefore never increases RSS; it falls whenever the new term explains any part of the residual, and stays the same only in the degenerate case where the new term is exactly uncorrelated with the current residuals.

That is why RSS (and \(R^{2}\)) cannot be used alone to choose a model, and adjusted \(R^{2}\) or an F-test is needed. The statements saying RSS increases are wrong in every case, and the condition 'provided the response is dependent on the new term' is unnecessary: even an irrelevant term lowers RSS by chance.

Discuss in Forum

Let \(X_{1}\), \(X_{2}\), .......... \(X_{n}\) be iid r.v.s satisfying the following regression equation :

\(X_{i} = \alpha z_{i} + e_{i}\); \(i = 1, 2, ....... n\)

where \(z_{1}\), \(z_{2}\), ....... \(z_{n}\) are fixed and \(e_{i}\)'s \((i = 1, 2, ..... n)\) are iid r.v.s with \(N(0, \sigma^{2})\), \(\sigma^{2}\) is unknown MLE of \(\alpha\) is given as :

A.\(\dfrac{\sum_{1}^{n} x_{i}z_{i}}{\sum_{1}^{n} x_{i}^{2}}\)
B.\(\dfrac{\sum_{1}^{n} x_{i}z_{i}}{n}\)
C.\(\dfrac{\sum z_{i}^{2}}{n}\)
D.\(\dfrac{\sum x_{i}z_{i}}{\sum z_{i}^{2}}\)

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: D

The MLE is \(\hat{\alpha} = \sum x_{i}z_{i}/\sum z_{i}^{2}\).

With normal errors, maximising the likelihood in \(\alpha\) is the same as minimising \(\sum(x_{i} - \alpha z_{i})^{2}\), whatever \(\sigma^{2}\) is. Setting the derivative to zero:

\[ \begin{aligned} -2\sum z_{i}(x_{i} - \alpha z_{i}) &= 0 \\ \hat{\alpha} &= \frac{\sum x_{i}z_{i}}{\sum z_{i}^{2}} \end{aligned} \]

This is least squares through the origin. Option A divides by \(\sum x_{i}^{2}\), which would regress \(z\) on \(x\); option B divides by \(n\), which is correct only if every \(z_{i}^{2} = 1\); and option C does not involve the data at all.

Discuss in Forum

For a standard multiple regression model \(\{Y, X\beta, \sigma^{2}I\}\), which of the following statements is not true ?

A.\(E(\hat{Y}) = E(HY) = X\beta\), where \(H = X(X'X)^{-1}X'\)
B.\(Cov(\hat{Y}) = \sigma^{2}H\)
C.\(Cov(\hat{Y}, Y - \hat{Y}) = 0\)
D.\(\hat{Y} \sim N_{n}(\beta, \sigma^{2}H)\), under normality

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: D

The statement that is not true is \(\hat{Y} \sim N_{n}(\beta, \sigma^{2}H)\).

With \(H = X(X'X)^{-1}X'\) the hat matrix, \(\hat{Y} = HY\). Since \(HX = X\), \(E(\hat{Y}) = HX\beta = X\beta\) (A). Since H is symmetric and idempotent, \(Cov(\hat{Y}) = H\sigma^{2}IH' = \sigma^{2}H\) (B). And \(Cov(HY, (I - H)Y) = \sigma^{2}H(I - H) = 0\) (C).

Under normality \(\hat{Y}\) is normal with mean \(X\beta\), not \(\beta\). The vector \(\beta\) has \(p\) components while \(\hat{Y}\) has \(n\), so the stated mean is not even of the right size.

Discuss in Forum

Consider the multiple regression set-up \(\{Y, X\beta, \sigma^{2}I\}\) with \(p\)-regressors. The ordinary least squares estimator (OLSE) \(\hat{\beta}\) is the vector which minimizes \(E(Y - X\beta)(Y - X\beta)'\). Which of the following statements is not true under this set-up ?

A.\(X\hat{\beta}\) is always unique
B.\(\hat{\beta}\) is unique if and only if rank (X) = \(p\)
C.\(\hat{\beta}\) is unique if and only if rank (X) \(\lt p\)
D.OLSE \(K'\hat{\beta}\) of \(K'\beta\) is unique if and only if \(K'\beta\) is estimable

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: C

The statement that is not true is \(\hat{\beta}\) is unique if and only if rank(X) \(\lt p\).

The normal equations \(X'X\hat{\beta} = X'Y\) always have a solution, and the fitted vector \(X\hat{\beta}\) is always unique, being the orthogonal projection of Y onto the column space of X (A). The solution \(\hat{\beta}\) is unique exactly when \(X'X\) is invertible, that is when rank(X) = \(p\) (B). When rank(X) \(\lt p\) there are infinitely many solutions, which is the opposite of C.

Even then, a linear function \(K'\hat{\beta}\) is the same for every solution precisely when \(K'\beta\) is estimable, which is D.

Discuss in Forum

Which of the following statements is not true in the context of the transformation of a response variable under a regression set-up ?

A.When E(Y) = V(Y) = C (a constant), square root transformation is more appropriate
B.When E(Y) = C and \(V(Y) = C^{2}\), log transformation is more appropriate
C.Scale transformation preserve the directions of the association between Y and X
D.The Box-Cox transformation \(Y \to g(Y, \lambda)\) is a discontinuous function of \(\lambda\)

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: D

The statement that is not true is The Box-Cox transformation is a discontinuous function of \(\lambda\).

The Box-Cox transformation is \(g(Y, \lambda) = (Y^{\lambda} - 1)/\lambda\) for \(\lambda \ne 0\) and \(\log Y\) for \(\lambda = 0\). The \(-1\) and the division by \(\lambda\) are there precisely to make it continuous in \(\lambda\): as \(\lambda \to 0\), \((Y^{\lambda} - 1)/\lambda \to \log Y\).

The other statements are the standard variance-stabilising rules. When the variance equals the mean (Poisson-like data), \(\sqrt{Y}\) stabilises it (A). When the standard deviation is proportional to the mean, \(\log Y\) does (B). And a positive change of scale does not change the sign of the association (C).

Discuss in Forum

\(Y_{1}, Y_{2}\) and \(Y_{3}\) are three uncorrelated random variables with common variance \(\sigma^{2}\). Further \(E(Y_{1}) = \theta_{2} - \theta_{1}\), \(E(Y_{2}) = E(Y_{3}) = \theta_{2} + \theta_{3}\). Then :

A.\(\theta_{1} - \theta_{2}\) is not estimable
B.\(\theta_{1} + \theta_{2}\) is not estimable
C.\(\theta_{1} + \theta_{3}\) is not estimable
D.\(\theta_{2} + \theta_{3}\) is not estimable

Exam Year(s)

MH SET - 2019, SET Exam - 2019,

Answer: B

The true statement is \(\theta_{1} + \theta_{2}\) is not estimable.

A linear function is estimable when it is a linear combination of the expectations \(E(Y_{1}) = \theta_{2} - \theta_{1}\) and \(E(Y_{2}) = E(Y_{3}) = \theta_{2} + \theta_{3}\). For \(a(\theta_{2} - \theta_{1}) + b(\theta_{2} + \theta_{3})\) to equal \(\theta_{1} + \theta_{2}\), the \(\theta_{3}\) coefficient forces \(b = 0\), and then \(-a = 1\), \(a = 1\) contradict each other. So \(\theta_{1} + \theta_{2}\) is not estimable.

The others are estimable, so their 'not estimable' claims are false: \(\theta_{1} - \theta_{2} = -E(Y_{1})\); \(\theta_{2} + \theta_{3} = E(Y_{2})\); and \(\theta_{1} + \theta_{3} = E(Y_{2}) - E(Y_{1})\).

Discuss in Forum

In a simple linear regression of Y on X based on \(n\) observations. The fitted values of \(Y_{i}\)'s are \(Y_{i}^{n}\), \(i = 1 ..... n\) respectively. Then :

\[ \sum_{i=1}^{n} Y_{i} = \sum_{i=1}^{n} Y_{i}^{n} \]

A.always
B.only if the regression equation is \(Y = B_{0} + B_{1}X + \varepsilon\)
C.if and only if the regression equation is \(Y = B_{0} + B_{1}X + \varepsilon\)
D.if the regression equation is \(Y = B_{0} + B_{1}X + \varepsilon\)

Exam Year(s)

MH SET - 2019, SET Exam - 2019,

Answer: D

The equality holds if the regression equation is \(Y = B_{0} + B_{1}X + \varepsilon\).

The paper writes the fitted values as \(Y_{i}^{n}\). When the model has an intercept, the least squares normal equation for \(B_{0}\) is \(\sum(Y_{i} - \hat{Y}_{i}) = 0\): the residuals sum to zero, so \(\sum Y_{i} = \sum \hat{Y}_{i}\).

It does not hold 'always': for regression through the origin, \(Y = B_{1}X + \varepsilon\), the residuals need not sum to zero. And 'only if' is too strong, because any model with an intercept (for example a multiple regression) has the same property. So the condition is sufficient, not necessary, which is what option D says.

Discuss in Forum

A least squares regression fit :

A.may be used to predict the value of Y if the corresponding values of regressor X are given
B.indicates a cause-effect relationship between response variable Y and regressors
C.can be determined only if a satisfactory relationship exists between response variable Y and regression
D.all the above statements are true

Exam Year(s)

MH SET - 2019, SET Exam - 2019,

Answer: A

A least squares fit may be used to predict Y from given values of the regressor X.

That is its purpose: the fitted equation \(\hat{Y} = b_{0} + b_{1}X\) gives the predicted response for a given X (sensibly within the range of the data).

It does not show cause and effect: a regression measures association, and a strong fit can arise from a common cause or coincidence. Nor does fitting require a satisfactory relationship to exist: least squares produces a line for any data, however poor the fit. So B and C are false, and therefore D is too.

Discuss in Forum

Suppose \(Y_{i} = B_{0} + B_{1}(x_{i} - 2) + \varepsilon_{i}\), \(E(\varepsilon_{i}) = 0\), \(\mathrm{Var}(\varepsilon_{i}) = \sigma^{2}\) and where \(x_{i} = i\), \(i = 1, 2, 3\). The Best Linear Unbiased estimators of \(B_{0}\) and \(B_{1}\).

A.do not exist
B.exist and are uncorrelated with each other
C.exist and are independent of each other
D.exist and are positively correlated with each other

Exam Year(s)

MH SET - 2019, SET Exam - 2019,

Answer: B

The BLUEs exist and are uncorrelated with each other.

The regressor values are \(x_{i} - 2 = -1, 0, 1\), which sum to 0. The design matrix has full column rank, so the least squares estimators are the BLUEs (Gauss-Markov), and with a centred regressor

\[ \hat{B}_0 = \bar{Y},\ \ \hat{B}_1 = \frac{Y_3 - Y_1}{2} \]

Their covariance is \(-\sigma^{2}\bar{c}/S_{cc}\), where \(\bar{c} = 0\) is the mean of the centred regressor, so it is 0. Directly: \(\mathrm{Cov}(\bar{Y}, (Y_{3} - Y_{1})/2)\) is \(\tfrac{1}{6}(\sigma^{2} - \sigma^{2}) = 0\).

Independence would need normal errors, which are not assumed, so C goes too far; and they are not positively correlated.

Discuss in Forum

10.

The regression equation of Y on X is given by \(y = 2 - 1.2x\) and that of X on Y is given by \(x = -1 - 0.3y\). Hence the correlation coefficient between X and Y is :

A.+ 0.36
B.+ 0.6
C.- 0.6
D.- 0.36

Exam Year(s)

MH SET - 2020, SET Exam - 2020,

Answer: C

The correlation coefficient is -0.6.

The regression coefficients are \(b_{yx} = -1.2\) and \(b_{xy} = -0.3\), and \(r^{2} = b_{yx}b_{xy}\):

\[ r^{2} = (-1.2)(-0.3) = 0.36, \quad |r| = 0.6 \]

\(r\) has the same sign as the two regression coefficients, which are both negative, so \(r = -0.6\).

0.36 is \(r^{2}\), not \(r\), and +0.6 has the wrong sign.

Discuss in Forum

Loading…

Linear Models and Regression Questions and Answers

Topics

Other Subjects

Tags: Linear Models and Regression Questions and Answers

About Us

Linear Models and Regression Questions and Answers

Topics

Other Subjects

Tags: Linear Models and Regression Questions and Answers

About Us

Stay informed about all important Dates & Notifications