Multivariate Analysis Questions and Answers

Answer: A

\(X_{1}\) and \(X_{2}\) are negatively correlated.

\(X_{2}\) is again N(0, 1), by the symmetry of the normal distribution, and \(X_{1}X_{2} = -X_{1}^{2}\) on \(|X_{1}| \le 3\) and \(+X_{1}^{2}\) outside. So

\[ \begin{aligned} \mathrm{Cov} &= -E[X_{1}^{2}; |X_{1}| \le 3] \\ &\quad + E[X_{1}^{2}; |X_{1}| \gt 3] \end{aligned} \]

Almost all of \(E[X_{1}^{2}] = 1\) comes from \(|X_{1}| \le 3\) (the tail beyond 3 contributes about 0.03), so the covariance is about \(-0.97 + 0.03 \lt 0\).

The correlation is not \(-1\), because \(X_{2} = X_{1}\) on the tails, so they are not perfectly correlated. They are not jointly normal either: \(X_{1} + X_{2}\) equals 0 with probability about 0.997 but is not identically 0, which is impossible for a normal variable.

Answer: A

The singular pair is \(X_{1} + X_{3},\ X_{1} - X_{2}\).

With \(X_{3} = X_{1} - 2X_{2}\):

\[ X_{1} + X_{3} = 2X_{1} - 2X_{2} = 2(X_{1} - X_{2}) \]

so the two components of option A are exact multiples of each other. Their joint normal distribution lives on a line, and its covariance matrix has rank 1: it is singular.

The others are non-singular linear transforms of \((X_{1}, X_{2})\), whose covariance \(\Sigma\) is positive definite: \((X_{1} - X_{3}, X_{2} - X_{3}) = (2X_{2}, 3X_{2} - X_{1})\), \((X_{1}, X_{3})\) and \((X_{2}, X_{3})\) each come from a \(2 \times 2\) matrix with non-zero determinant.

Answer: A

\(E X_{1}X_{2}^{2}X_{3}\) is \((-1)^{4}\,\partial^{4}\phi_{X}/\partial t_{1}\partial t_{2}^{2}\partial t_{3}\) at \(\underline{t} = \underline{0}\).

Differentiating \(\phi_{X}(t) = E e^{i t'X}\) brings down a factor \(iX_{j}\) for each derivative in \(t_{j}\). A fourth-order mixed derivative at \(t = 0\) is \(i^{4}E[X_{1}X_{2}^{2}X_{3}]\), so the moment is \(i^{-4} = (-i)^{4}\) times the derivative at 0.

Since \((-i)^{4} = 1 = (-1)^{4}\), the expression in option A equals the moment. Moments come from derivatives at the origin only; the options evaluated at (1, 2, 1) give something else, and a third-order derivative cannot produce a fourth-order moment.

Answer: C

The covariance matrix is \(\begin{bmatrix} 2 & -2 \\ -2 & 3 \end{bmatrix}\).

A covariance matrix must be symmetric and positive semi-definite. For a 2 by 2 symmetric matrix this means non-negative diagonal entries and a non-negative determinant. For option C, the diagonal is 2 and 3 and

\[ \det = 2 \cdot 3 - (-2)^{2} = 2 \gt 0 \]

Option A has determinant \(0 - 1/4 \lt 0\). Option B is not symmetric. Option D has determinant \(4 - 9 \lt 0\), which would mean a correlation of -3/2, outside [-1, 1].

Answer: A

The distribution is univariate normal.

Each \(a_{i}\) is a \(1 \times p\) row vector and each \(Y_{i}\) is a \(p \times 1\) normal vector, so \(a_{i}Y_{i}\) is a scalar, a linear combination of the components of \(Y_{i}\). A linear combination of a multivariate normal vector is univariate normal, and the three terms are independent, so their sum is univariate normal with mean \(\sum a_{i}\mu\) and variance \(\sum a_{i}\Sigma a_{i}^{t}\).

The result is a single number, not a vector, so it cannot be trivariate or four-variate; and linear functions of normal vectors are always normal.

Answer: B

The statement that is not correct is \(Y_{1}, Y_{2}, Y_{3}, Y_{4}\) are identically normally distributed.

The principal components are \(Y_{i} = e_{i}'X\) for orthonormal eigenvectors \(e_{i}\) of \(\Sigma\). Each is normal with variance \(\lambda_{i}\), and their covariances are 0; jointly normal and uncorrelated means independent, so (A) is true.

But the variances 6, 4, 3, 2 differ, so the components are not identically distributed. The others are true: the first two explain \((6 + 4)/15 \approx 66.7\%\) of the total variance 15, more than 60%, and \(Y_{4}\) has the smallest eigenvalue, 2.

Answer: B

The mean and variance are \(k\) and \(2k\).

Reading the printed \(X^{1}\) as the transpose \(X'\), \(Z = X'BX\) with \(B = \Sigma^{-1}A\). For \(X \sim N(0, \Sigma)\),

\[ \begin{aligned} E(Z) &= \mathrm{tr}(B\Sigma) = \mathrm{tr}(\Sigma^{-1}A\Sigma) \\ &= \mathrm{tr}(A) = \mathrm{rank}(A) = k \end{aligned} \]

since an idempotent matrix has trace equal to its rank. Also \((B\Sigma)^{2} = \Sigma^{-1}A^{2}\Sigma = B\Sigma\), so \(B\Sigma\) is idempotent of rank \(k\), and (with B symmetric, as the standard result assumes) Z is distributed as chi-square with \(k\) degrees of freedom, whose variance is \(2\,\mathrm{tr}(B\Sigma)^{2} = 2k\). Options in \(n\) use the dimension instead of the rank, and 'k, k' gives the right mean but a chi-square variable's variance is twice its degrees of freedom.

Answer: C

The conditional parameters are \(\mu^{*} = (\mu_{1} + \rho^{2}(x_{3} - \mu_{3}),\ \mu_{2})\) and \(\Sigma^{*}\) with rows \((1 - \rho^{4}, \rho)\), \((\rho, 1)\).

Partition \(\Sigma\) with \(\Sigma_{12} = (\rho^{2}, 0)'\) (the covariances of \(X_{1}, X_{2}\) with \(X_{3}\)) and \(\Sigma_{22} = 1\). Then

\[ \begin{aligned} \mu^{*} &= \mu_{(1)} + \Sigma_{12}(x_{3} - \mu_{3}) \\ \Sigma^{*} &= \Sigma_{11} - \Sigma_{12}\Sigma_{12}' \end{aligned} \]

The mean of \(X_{1}\) shifts by \(\rho^{2}(x_{3} - \mu_{3})\) and \(X_{2}\), uncorrelated with \(X_{3}\), keeps mean \(\mu_{2}\). Only the (1, 1) entry of the covariance changes, to \(1 - \rho^{4}\). Options A and B keep the unconditional covariance, and options A and D the unconditional mean.

Answer: C

The distribution is trivariate normal.

Each \(X_{i}\) is a 3-dimensional normal vector, and they are independent, so the stacked 12-dimensional vector is multivariate normal. The combination \(\sum a_{i}X_{i}\) is a linear transformation of it into \(\mathbf{R}^{3}\), and linear transformations of multivariate normal vectors are normal. Its mean is \(\sum a_{i}\mu\) and its covariance \((\sum a_{i}^{2})\Sigma\).

The result is still a 3-component vector, so it is neither univariate nor four-variate; the number 4 counts the vectors summed, not the dimension. And it is normal, not 'not normal'.

Answer: B

The possible conditional distribution is N(2.6, 1.84).

For a bivariate normal, \(E(X_{1} \mid X_{2} = x)\) \(= \mu_{1} + \rho\frac{\sigma_{1}}{\sigma_{2}}(x - \mu_{2})\) and \(V(X_{1} \mid X_{2}) = \sigma_{1}^{2}(1 - \rho^{2})\). Here \(\mu_{1} = 3\), \(\mu_{2} = 4\), \(\sigma_{1}^{2} = 2\) and \(\rho \gt 0\). Since \(x - \mu_{2} = 2 - 4 = -2 \lt 0\), the conditional mean is below 3, and the conditional variance is below 2.

Only N(2.6, 1.84) satisfies both: mean less than 3 and variance less than 2. N(1.5, 2.64) and N(3.5, 2.84) have variance above 2, and N(3.5, 1.52) has a mean above 3, which would need negative correlation.