Multivariate Analysis Questions and Answers

Answer: C

The partial correlation is -0.5.

\[ \begin{aligned} r_{23.1} &= \frac{r_{23} - r_{21}r_{31}}{\sqrt{(1 - r_{21}^{2})(1 - r_{31}^{2})}} \\ &= \frac{0.24 - 0.6 \times 0.8}{\sqrt{0.64 \times 0.36}} \\ &= \frac{-0.24}{0.48} = -0.5 \end{aligned} \]

Although \(X_{2}\) and \(X_{3}\) are positively correlated overall, most of that comes from their shared dependence on \(X_{1}\). Once \(X_{1}\) is held fixed, the remaining association is negative.

\(\pm 0.24\) simply reuse the raw correlation \(r_{23}\) and ignore \(X_{1}\), and +0.5 has the sign of the numerator wrong.

Answer: C

The dimension reduction technique is principal component analysis.

PCA replaces \(p\) correlated variables by a few uncorrelated linear combinations, the principal components, which are the eigenvectors of the covariance matrix ordered by eigenvalue (variance explained). Keeping the first \(k\) components summarises the data in \(k \lt p\) dimensions with the least loss of variance. It uses no group labels.

Linear and quadratic discriminant analysis are classification methods: they build rules to assign an observation to one of known groups. Fisher's linear discriminant does project data onto a few directions as a by-product, but its purpose is separating labelled groups, and QDA does not reduce dimension at all. PCA is the technique designed for dimension reduction, so 'none of the above' is wrong too.

Answer: A

The correlation is 1/(2√2).

Read the variances and covariances from the matrix: \(Var(X_{1}) = 2\), \(Var(X_{2}) = 1\), \(Var(X_{3}) = 4\), \(Cov(X_{1}, X_{2}) = 0.5\), \(Cov(X_{1}, X_{3}) = 2\), \(Cov(X_{2}, X_{3}) = 0\).

\[ \begin{aligned} Cov(X_{1} - X_{3}, 2X_{2}) &= 2(0.5 - 0) = 1 \\ Var(X_{1} - X_{3}) &= 2 + 4 - 2(2) = 2 \\ Var(2X_{2}) &= 4 \end{aligned} \]

\[ \rho = \frac{1}{\sqrt{2 \times 4}} = \frac{1}{2\sqrt{2}} \]

\(1/\sqrt{2}\) forgets the factor 2 in \(2X_{2}\) when computing the variance, and 0 would require the covariance to vanish.

Answer: C

The correct statement is E(Y | X = x) = E(Y) + a linear function of x.

For a bivariate normal,

\[ \begin{aligned} E(Y \mid X = x) &= \mu_{Y} \\ &\quad + \rho\frac{\sigma_{Y}}{\sigma_{X}}(x - \mu_{X}) \end{aligned} \]

which is E(Y) plus a linear function of x. The conditional variance is \(\sigma_{Y}^{2}(1 - \rho^{2})\), a constant that does not depend on x at all.

So the options making the conditional variance depend on x (through \(x^{2}\) or a quadratic) are wrong, and \(x\,E(Y)\) is not the form of the regression function.

Answer: D

The distribution cannot be determined from the given information.

If A is symmetric with all eigenvalues 1, then A = I and \(Y = \underline{X}'\underline{X}\). Its distribution depends on \(\underline{\mu}\) and \(\Sigma\), which are not given. Y is \(\chi^{2}_{3}\) only when \(\underline{\mu} = \underline{0}\) and \(\Sigma = I\); with a non-zero mean it is non-central chi-square, and with a general \(\Sigma\) it is a weighted sum of chi-squares.

A quadratic form in normal variables is never normal, which rules out option B, and chi-square with 1 degree of freedom would need A of rank 1. So no specific distribution follows.

Answer: A

The quadratic form is positive definite.

Its symmetric matrix is

\[ A = \begin{pmatrix} 1 & -\frac{1}{2} & \frac{1}{2} \\ -\frac{1}{2} & 2 & 0 \\ \frac{1}{2} & 0 & 4 \end{pmatrix} \]

The leading principal minors are 1, \(2 - \frac{1}{4} = \frac{7}{4}\), and

\[ \det A = 8 - 1 - \tfrac{1}{2} = \tfrac{13}{2} \]

All three are positive, so by Sylvester's criterion the form is positive definite. Positive semidefinite would allow a zero minor, and the negative cases need the minors to alternate in sign.

Answer: D

The partial correlation is the correlation between \(X_{1} - \hat{X}_{1}\) and \(X_{2} - \hat{X}_{2}\).

Removing the effect of \(X_{3}\) means regressing \(X_{1}\) and \(X_{2}\) on \(X_{3}\) and keeping the residuals \(X_{1} - \hat{X}_{1}\) and \(X_{2} - \hat{X}_{2}\), the parts not explained by \(X_{3}\). The partial correlation \(r_{12.3}\) is the simple correlation between these residuals.

Correlating \(X_{1}\) with \(\hat{X}_{2}\), or \(\hat{X}_{1}\) with \(X_{2}\), mixes in the part explained by \(X_{3}\). Subtracting \(X_{3}\) itself, as in \(X_{1} - X_{3}\), removes \(X_{3}\) with coefficient 1 instead of the fitted regression coefficient.

Answer: A

The distance is 5.

The difference of the two vectors is \((2 - 2, -2 - 1, 2 - (-2)) = (0, -3, 4)\), so

\[ d = \sqrt{0^{2} + (-3)^{2} + 4^{2}} = \sqrt{25} = 5 \]

4 is just the largest coordinate difference, 2 adds the differences with their signs, and 0 would mean the point is the cluster centre.

Answer: D

Principal component analysis is used for reducing the number of variables in the data.

PCA replaces the original correlated variables by uncorrelated linear combinations, the principal components, ordered by the variance they explain. Keeping the first few components retains most of the total variability with far fewer variables, which is dimension reduction.

PCA keeps all the observations. It aims to preserve the variability, not to reduce it. Expressing one variable as a function of others is regression, not PCA.

Answer: D

The conditional variance is \(2(1 + \rho)\).

Let \(U = X_{1} + X_{2}\) and \(V = X_{1} - X_{2}\). They are jointly normal with

\[ \begin{aligned} \mathrm{Cov}(U, V) &= \mathrm{Var}(X_{1}) - \mathrm{Var}(X_{2}) = 0 \\ \mathrm{Var}(U) &= 1 + 1 + 2\rho = 2(1 + \rho) \end{aligned} \]

Uncorrelated jointly normal variables are independent, so conditioning on V does not change the distribution of U, and \(\mathrm{var}(U \mid V) = 2(1 + \rho)\).

\(1 + \rho\) is half the answer, 2 ignores the covariance term, and \(\rho(1 + \rho)\) does not come from any variance formula here.