Linear Models and Regression Questions and Answers

Q: In a multiple linear regression model \(y = X\beta + \varepsilon\), the covariance between y and \(\hat{y}\), the predicted value of y using least square estimator of \(\beta\) is :

The covariance is Σ X(X'X)−1X'.The fitted values are \(\hat{y} = Hy\) with \(H = X(X'X)^{-1}X'\). Writing \(\Sigma = Cov(y) = Cov(\varepsilon)\),\[ Cov(y, \hat{y}) = Cov(y, Hy) = \Sigma H' = \Sigma H \]since H is symmetric. So the answer is \(\Sigma X(X'X)^{-1}X'\); with \(\Sigma = \sigma^{2}I\) this is \(\sigma^{2}H\).\(\Sigma(I - H)\) is the covariance between y and the residuals, and \(\Sigma\) alone is the covariance of y with itself.

Q: If \(e_{1}, e_{2}, \ldots, e_{n}\) are the residuals obtained on fitting a simple linear regression \(y = \beta_{0} + \beta_{1}X + \varepsilon\) to \((Y_{i}, X_{i})\) \(i = 1 \ldots n\) under the standard assumption, then :

\(\sum X_{i}e_{i}\) is always zero.The least squares normal equations are obtained by setting the derivatives of \(\sum(Y_{i} - \beta_{0} - \beta_{1}X_{i})^{2}\) to zero:\[ \begin{aligned} \sum e_{i} &= 0 \\ \sum X_{i}e_{i} &= 0 \end{aligned} \]The second equation is exactly the statement that the residuals are orthogonal to the regressor. It holds for every data set fitted by least squares, so the sum is never positive or negative and something definite can be said about it.

Q: Consider a logistic regression model consisting of only one explanatory variable X which takes values in {−1, 1}. Then the odds ratio on changing X from −1 to 1 is :

The odds ratio is exp(2β).In the logistic model \(\log\frac{p}{1 - p} = \alpha + \beta X\), the odds are \(e^{\alpha + \beta X}\). Comparing X = 1 with X = −1:\[ \frac{e^{\alpha + \beta}}{e^{\alpha - \beta}} = e^{2\beta} \]The change in X is 2 units, not 1, so the familiar \(\exp(\beta)\), which is the odds ratio for a one-unit change, is not the answer here. \(\exp(1 \pm \beta)\) mixes the value of X into the exponent wrongly.

Q: In a linear model \(Y_{4 \times 1} = X_{4 \times 3}\,\theta_{3 \times 1} + \epsilon_{4 \times 1}\) \(E(Y_{1}) = E(Y_{3}) = \theta_{1} + \theta_{2}\) and \(E(Y_{2}) = E(Y_{4}) = \theta_{1} - \theta_{3}\). Hence which of the following statement is true ?

The estimable function is \(\theta_{2} + \theta_{3}\).A linear function is estimable exactly when it is a linear combination of the expected observations. Here those are \(\theta_{1} + \theta_{2}\) and \(\theta_{1} - \theta_{3}\), so estimable functions have the form\[ \begin{aligned} &a(\theta_{1} + \theta_{2}) + b(\theta_{1} - \theta_{3}) \\ &= (a + b)\theta_{1} + a\theta_{2} - b\theta_{3} \end{aligned} \]For \(\theta_{2} + \theta_{3}\): \(a = 1\), \(b = -1\), \(a + b = 0\). It works, and \(E(Y_{1} - Y_{2}) = \theta_{2} + \theta_{3}\).For \(\theta_{1} + \theta_{2} + \theta_{3}\): \(a = 1\), \(b = -1\) gives coefficient 0 for \(\theta_{1}\), not 1. For \(\theta_{1} + \theta_{3}\): \(a = 0\), \(b = -1\) gives \(-1\) for \(\theta_{1}\). The individual \(\theta\)'s are not estimable, since the design matrix has rank 2.

Q: In case of ANOVA with standard assumption, the test of hypothesis tests :

ANOVA tests equality of means when variances are all equal.The standard assumptions are independent, normally distributed errors with a common variance. Under them, the F test compares between-group and within-group mean squares to test \(H_{0}: \mu_{1} = \mu_{2} = \ldots = \mu_{k}\). Equal variances are an assumption of the test, not something it tests.'Equality of means' alone omits the homogeneity condition the F distribution relies on. Testing equality of variances is the job of tests such as Bartlett's or Levene's, and ANOVA does not test means and variances together.

Q: In a simple linear regression, the prediction of the response variable y.

The prediction will have larger variation as the distance between X and \(\overline{X}\) increases.In simple linear regression the variance of the predicted mean response at \(x_{0}\) is\[ \begin{aligned} &\mathrm{Var}(\hat{y}_{0}) \\ &= \sigma^{2}\left(\frac{1}{n} + \frac{(x_{0} - \overline{x})^{2}}{S_{xx}}\right) \end{aligned} \](and a new observation adds a further \(\sigma^{2}\)). The variance is smallest at \(x_{0} = \overline{x}\) and grows with \((x_{0} - \overline{x})^{2}\), which is why prediction bands flare out away from the mean.The prediction does depend on X, its variation is not the same for all X, and it grows with distance from \(\overline{X}\) in either direction, not simply as X increases.

Q: In case of a multiple linear regression setup \(\underline{Y} = X\underline{B} + \underline{\varepsilon}\) with \(\mathrm{cov}(\underline{\varepsilon}) = \sigma^{2} I_{n}\), \(\sigma^{2} \gt 0\). Let MReg ss MError ss and MTotal ss stand for mean regression sum of squares, mean error sum of squares and mean total sum of squares respectively. Which of the following statements is true ?

The true statement is that MError ss is unbiased for \(\sigma^{2}\), while MReg ss and MTotal ss are biased.The residual mean square \(\frac{SSE}{n - p}\) has expectation exactly \(\sigma^{2}\) under the model, whatever the coefficients.The regression mean square has expectation \(\sigma^{2}\) plus a non-negative term that depends on the slope coefficients, and that term is zero only when the slopes are all zero. The total mean square mixes the two and is biased for the same reason.So options A, B and C, which call MReg ss unbiased or MError ss biased, are wrong.

Q: In a multiple linear regression setup, the adjusted coefficient of determination accounts for :

The adjusted coefficient of determination accounts for the number of regressors in the model.\[ \bar{R}^{2} = 1 - (1 - R^{2})\frac{n - 1}{n - k - 1} \]where k is the number of regressors. Ordinary \(R^{2}\) never falls when a variable is added, even a useless one; the adjustment penalises extra regressors, so \(\bar{R}^{2}\) rises only when a new variable improves the fit by more than chance would.It does nothing special about outliers or unusually large fitted values ('valves' in the paper), which are handled by residual and influence diagnostics.

Q: Consider a logistic regression modelE(Y) = exp(B0 + B1X1 + B2X2 + B3X3 + B4X4) / [1 + exp(B0 + B1X1 + B2X2 + B3X3 + B4X4)]For the deviance based test to test whether the model fits the data of 40 observations, the degrees of freedom are :

The deviance test has 35 degrees of freedom.The model has five parameters, \(B_{0}, B_{1}, B_{2}, B_{3}, B_{4}\). The deviance compares the fitted model with the saturated model, which has one parameter per observation, so its degrees of freedom are\[ n - p = 40 - 5 = 35 \]34 subtracts one parameter too many, 5 is the number of parameters, and 4 is the number of predictors, which is the degrees of freedom for testing all four slopes against the intercept-only model, not for goodness of fit.

31.

In a multiple linear regression model \(y = X\beta + \varepsilon\), the covariance between y and \(\hat{y}\), the predicted value of y using least square estimator of \(\beta\) is :

A.\(\Sigma X(X'X)^{-1}X'\)
B.\(\Sigma(I - X(X'X)^{-1}X')\)
C.\(\Sigma\)
D.None of the above

Exam Year(s)

MH SET - 2024, SET Exam - 2024,

Answer: A

The covariance is Σ X(X'X)⁻¹X'.

The fitted values are \(\hat{y} = Hy\) with \(H = X(X'X)^{-1}X'\). Writing \(\Sigma = Cov(y) = Cov(\varepsilon)\),

\[ Cov(y, \hat{y}) = Cov(y, Hy) = \Sigma H' = \Sigma H \]

since H is symmetric. So the answer is \(\Sigma X(X'X)^{-1}X'\); with \(\Sigma = \sigma^{2}I\) this is \(\sigma^{2}H\).

\(\Sigma(I - H)\) is the covariance between y and the residuals, and \(\Sigma\) alone is the covariance of y with itself.

Discuss in Forum

32.

If \(e_{1}, e_{2}, \ldots, e_{n}\) are the residuals obtained on fitting a simple linear regression \(y = \beta_{0} + \beta_{1}X + \varepsilon\) to \((Y_{i}, X_{i})\) \(i = 1 \ldots n\) under the standard assumption, then :

A.\(\sum_{i=1}^{n} X_{i}e_{i}\) is always positive
B.\(\sum_{i=1}^{n} X_{i}e_{i}\) is always negative
C.\(\sum_{i=1}^{n} X_{i}e_{i}\) is always zero
D.Nothing can be said about \(\sum_{i=1}^{n} X_{i}e_{i}\)

Exam Year(s)

MH SET - 2024, SET Exam - 2024,

Answer: C

\(\sum X_{i}e_{i}\) is always zero.

The least squares normal equations are obtained by setting the derivatives of \(\sum(Y_{i} - \beta_{0} - \beta_{1}X_{i})^{2}\) to zero:

\[ \begin{aligned} \sum e_{i} &= 0 \\ \sum X_{i}e_{i} &= 0 \end{aligned} \]

The second equation is exactly the statement that the residuals are orthogonal to the regressor. It holds for every data set fitted by least squares, so the sum is never positive or negative and something definite can be said about it.

Discuss in Forum

33.

Consider a logistic regression model consisting of only one explanatory variable X which takes values in {−1, 1}. Then the odds ratio on changing X from −1 to 1 is :

A.\(\exp(2\beta)\)
B.\(\exp(\beta)\)
C.\(\exp(1 + \beta)\)
D.\(\exp(1 - \beta)\)

Exam Year(s)

MH SET - 2024, SET Exam - 2024,

Answer: A

The odds ratio is exp(2β).

In the logistic model \(\log\frac{p}{1 - p} = \alpha + \beta X\), the odds are \(e^{\alpha + \beta X}\). Comparing X = 1 with X = −1:

\[ \frac{e^{\alpha + \beta}}{e^{\alpha - \beta}} = e^{2\beta} \]

The change in X is 2 units, not 1, so the familiar \(\exp(\beta)\), which is the odds ratio for a one-unit change, is not the answer here. \(\exp(1 \pm \beta)\) mixes the value of X into the exponent wrongly.

Discuss in Forum

34.

In a linear model \(Y_{4 \times 1} = X_{4 \times 3}\,\theta_{3 \times 1} + \epsilon_{4 \times 1}\) \(E(Y_{1}) = E(Y_{3}) = \theta_{1} + \theta_{2}\) and \(E(Y_{2}) = E(Y_{4}) = \theta_{1} - \theta_{3}\). Hence which of the following statement is true ?

A.\(\theta_{1}, \theta_{2}, \theta_{3}\) are all estimable
B.\(\theta_{1} + \theta_{2} + \theta_{3}\) is estimable
C.\(\theta_{1} + \theta_{3}\) is estimable
D.\(\theta_{2} + \theta_{3}\) is estimable

Exam Year(s)

MH SET - 2025, SET Exam - 2025,

Answer: D

The estimable function is \(\theta_{2} + \theta_{3}\).

A linear function is estimable exactly when it is a linear combination of the expected observations. Here those are \(\theta_{1} + \theta_{2}\) and \(\theta_{1} - \theta_{3}\), so estimable functions have the form

\[ \begin{aligned} &a(\theta_{1} + \theta_{2}) + b(\theta_{1} - \theta_{3}) \\ &= (a + b)\theta_{1} + a\theta_{2} - b\theta_{3} \end{aligned} \]

For \(\theta_{2} + \theta_{3}\): \(a = 1\), \(b = -1\), \(a + b = 0\). It works, and \(E(Y_{1} - Y_{2}) = \theta_{2} + \theta_{3}\).

For \(\theta_{1} + \theta_{2} + \theta_{3}\): \(a = 1\), \(b = -1\) gives coefficient 0 for \(\theta_{1}\), not 1. For \(\theta_{1} + \theta_{3}\): \(a = 0\), \(b = -1\) gives \(-1\) for \(\theta_{1}\). The individual \(\theta\)'s are not estimable, since the design matrix has rank 2.

Discuss in Forum

35.

In case of ANOVA with standard assumption, the test of hypothesis tests :

A.equality of means
B.equality of means when variances are all equal
C.equality of variances when means are all equal
D.equality of means and variances

Exam Year(s)

MH SET - 2025, SET Exam - 2025,

Answer: B

ANOVA tests equality of means when variances are all equal.

The standard assumptions are independent, normally distributed errors with a common variance. Under them, the F test compares between-group and within-group mean squares to test \(H_{0}: \mu_{1} = \mu_{2} = \ldots = \mu_{k}\). Equal variances are an assumption of the test, not something it tests.

'Equality of means' alone omits the homogeneity condition the F distribution relies on. Testing equality of variances is the job of tests such as Bartlett's or Levene's, and ANOVA does not test means and variances together.

Discuss in Forum

36.

In a simple linear regression, the prediction of the response variable y.

A.does not depend on the regression X
B.will have the same variation irrespective of the values of X
C.will have larger variation as values of X increase
D.will have larger variation as the distance between the regressor X and its mean \(\overline{X}\) increases

Exam Year(s)

MH SET - 2025, SET Exam - 2025,

Answer: D

The prediction will have larger variation as the distance between X and \(\overline{X}\) increases.

In simple linear regression the variance of the predicted mean response at \(x_{0}\) is

\[ \begin{aligned} &\mathrm{Var}(\hat{y}_{0}) \\ &= \sigma^{2}\left(\frac{1}{n} + \frac{(x_{0} - \overline{x})^{2}}{S_{xx}}\right) \end{aligned} \]

(and a new observation adds a further \(\sigma^{2}\)). The variance is smallest at \(x_{0} = \overline{x}\) and grows with \((x_{0} - \overline{x})^{2}\), which is why prediction bands flare out away from the mean.

The prediction does depend on X, its variation is not the same for all X, and it grows with distance from \(\overline{X}\) in either direction, not simply as X increases.

Discuss in Forum

37.

In case of a multiple linear regression setup \(\underline{Y} = X\underline{B} + \underline{\varepsilon}\) with \(\mathrm{cov}(\underline{\varepsilon}) = \sigma^{2} I_{n}\), \(\sigma^{2} \gt 0\). Let MReg ss MError ss and MTotal ss stand for mean regression sum of squares, mean error sum of squares and mean total sum of squares respectively. Which of the following statements is true ?

A.MReg ss and MError ss are unbiased estimators of \(\sigma^{2}\), MTotal ss is a biased estimator of \(\sigma^{2}\)
B.all three mean ss are unbiased estimators of \(\sigma^{2}\)
C.all three mean ss are biased estimator of \(\sigma^{2}\)
D.MError ss is an unbiased estimator of \(\sigma^{2}\), MReg ss and MTotal ss are biased estimator of \(\sigma^{2}\)

Exam Year(s)

MH SET - 2025, SET Exam - 2025,

Answer: D

The true statement is that MError ss is unbiased for \(\sigma^{2}\), while MReg ss and MTotal ss are biased.

The residual mean square \(\frac{SSE}{n - p}\) has expectation exactly \(\sigma^{2}\) under the model, whatever the coefficients.

The regression mean square has expectation \(\sigma^{2}\) plus a non-negative term that depends on the slope coefficients, and that term is zero only when the slopes are all zero. The total mean square mixes the two and is biased for the same reason.

So options A, B and C, which call MReg ss unbiased or MError ss biased, are wrong.

Discuss in Forum

38.

In a multiple linear regression setup, the adjusted coefficient of determination accounts for :

A.the outliers in the data
B.the unusually large fitted valves
C.the number of regressors in the model
D.none of the above

Exam Year(s)

MH SET - 2025, SET Exam - 2025,

Answer: C

The adjusted coefficient of determination accounts for the number of regressors in the model.

\[ \bar{R}^{2} = 1 - (1 - R^{2})\frac{n - 1}{n - k - 1} \]

where k is the number of regressors. Ordinary \(R^{2}\) never falls when a variable is added, even a useless one; the adjustment penalises extra regressors, so \(\bar{R}^{2}\) rises only when a new variable improves the fit by more than chance would.

It does nothing special about outliers or unusually large fitted values ('valves' in the paper), which are handled by residual and influence diagnostics.

Discuss in Forum

39.

Consider a logistic regression model

E(Y) = exp(B₀ + B₁X₁ + B₂X₂ + B₃X₃ + B₄X₄) / [1 + exp(B₀ + B₁X₁ + B₂X₂ + B₃X₃ + B₄X₄)]

For the deviance based test to test whether the model fits the data of 40 observations, the degrees of freedom are :

A.35
B.34
C.5
D.4

Exam Year(s)

MH SET - 2025, SET Exam - 2025,

Answer: A

The deviance test has 35 degrees of freedom.

The model has five parameters, \(B_{0}, B_{1}, B_{2}, B_{3}, B_{4}\). The deviance compares the fitted model with the saturated model, which has one parameter per observation, so its degrees of freedom are

\[ n - p = 40 - 5 = 35 \]

34 subtracts one parameter too many, 5 is the number of parameters, and 4 is the number of predictors, which is the degrees of freedom for testing all four slopes against the intercept-only model, not for goodness of fit.

Linear Models and Regression Questions and Answers

Topics

Other Subjects

Tags: Linear Models and Regression Questions and Answers

About Us

Linear Models and Regression Questions and Answers

Topics

Other Subjects

Tags: Linear Models and Regression Questions and Answers

About Us

Stay informed about all important Dates & Notifications