Markov Chains and Stochastic Processes Questions and Answers

Answer: D

All states are transient or persistent null (in fact null persistent).

This is a symmetric random walk on \(\{0, 1, 2, \ldots\}\), with the step down from 0 replaced by staying put. It behaves like the distance from the origin of a simple symmetric walk, which returns with probability 1, so every state is persistent.

A stationary distribution would satisfy \(\pi_{0} = \tfrac{1}{2}\pi_{0} + \tfrac{1}{2}\pi_{1}\) and \(\pi_{j} = \tfrac{1}{2}\pi_{j-1} + \tfrac{1}{2}\pi_{j+1}\), giving \(\pi_{j} = \pi_{0}\) for all \(j\), which cannot sum to 1. So there is no stationary distribution, and the persistent states are null.

Hence no state is non-null persistent, and no stationary distribution exists, ruling out A, B and C.

Answer: D

The extinction probability equals 1.

The sum is printed from \(\theta\), meaning from generation 0. With mean offspring \(m\), \(E(X_{n}) = m^{n}\) when \(X_{0} = 1\), so

\[ E\left(\sum_{n} X_{n}\right) = \sum_{n} m^{n} \]

which is finite only if \(m \lt 1\). A subcritical branching process dies out with probability 1. (Equivalently, a finite expected total progeny forces the total progeny to be finite almost surely, so the population cannot survive forever.)

An extinction probability in (0, 1) arises only for \(m \gt 1\), when the expected total is infinite; 0 is impossible once there is any chance of having no offspring, and the extinction probability always exists as a limit of \(P(X_{n} = 0)\).

Answer: A

The answer is both \(\{X_{n}\}\) and \(\{W_{n}\}\) are \(\mathbf{F}_{n}\)-martingales.

\(X_{n+1} = X_{n} + Y_{n+1}\), and \(Y_{n+1}\) is independent of \(\mathbf{F}_{n}\) with mean 0, so \(E[X_{n+1} \mid \mathbf{F}_{n}] = X_{n}\).

\(W_{n}\) is \(\mathbf{F}_{n}\)-measurable, so

\[ \begin{aligned} E[W_{n+1} \mid \mathbf{F}_{n}] &= W_{n} + W_{n}^{2}E[Y_{n+1}] \\ &= W_{n} \end{aligned} \]

(assuming the variables are integrable). But \(T_{n} = X_{n} + n\), so \(E[T_{n+1} \mid \mathbf{F}_{n}] = T_{n} + 1\): it is a submartingale, not a martingale. That rules out the options including \(T_{n}\), and 'only \(X_{n}\)' misses \(W_{n}\).

Answer: A

The statement that is not true is the forecast mean square error approaches \(\mu^{2} + \sigma^{2}/(1 - \phi^{2})\).

For a stationary AR(1) with constant, the \(k\)-step forecast is

\[ \begin{aligned} P(X_{t+k}) &= m + \phi^{k}(X_{t} - m), \\ m &= \frac{\mu}{1 - \phi} \end{aligned} \]

a weighted average of the mean \(E(X_{t}) = m\) and the latest value \(X_{t}\) (D), which tends to \(m\) as \(k \to \infty\) (C). Its error variance is \(\sigma^{2}(1 - \phi^{2k})/(1 - \phi^{2})\), which tends to the process variance \(\sigma^{2}/(1 - \phi^{2})\) (B).

A forecast error has mean 0, so the constant \(\mu\) never adds \(\mu^{2}\) to the mean square error; (A) is the false statement.

Answer: D

\(P[X_{1} = 1] = \) 1/6.

Multiply the initial distribution by the column of the transition matrix for state 1:

\[ \begin{aligned} P[X_{1} = 1] &= \tfrac{1}{4}\cdot\tfrac{1}{3} + \tfrac{1}{4}\cdot\tfrac{1}{3} + \tfrac{1}{2}\cdot 0 \\ &= \tfrac{1}{12} + \tfrac{1}{12} = \tfrac{1}{6} \end{aligned} \]

1/3 is just the transition probability from 0 (or 1) to 1, ignoring the starting distribution, and 1/4 is the initial probability of state 1.

Answer: A

\(E(T_{1} \mid X(T) = n) = \) \(\dfrac{T}{n + 1}\).

For a (homogeneous) Poisson process, given that \(n\) events occurred in [0, T], the event times are distributed as the order statistics of \(n\) independent \(U(0, T)\) variables. The smallest of \(n\) uniforms on [0, T] has mean

\[ E(U_{(1)}) = \frac{T}{n + 1} \]

(the \(n\) points split [0, T] into \(n + 1\) gaps of equal expected length). The answer does not depend on \(\lambda\), which rules out \(T\lambda^{n}\), and \(n/(n + 1) - T\) can be negative.

Answer: C

\(E\left(\sum X_{n}\right) = \) 2.

In a branching process with \(X_{0} = 1\), \(E(X_{n}) = \mu^{n}\). With \(\mu = 1/2\), the expected total progeny is

\[ \begin{aligned} \sum_{n=0}^{\infty} \left(\frac{1}{2}\right)^{n} &= \frac{1}{1 - 1/2} \\ &= 2 \end{aligned} \]

(by monotone convergence the expectation and the sum can be exchanged). The process is subcritical and dies out, so the total is finite, not \(\infty\). 1 counts only the ancestor and \(1\frac{1}{2}\) stops after the first generation.

Answer: C

The statement that is not true is \(|\gamma(k)| \ge \gamma(0)\).

By the Cauchy-Schwarz inequality, \(|\gamma(k)| = |\mathrm{Cov}(X_{t}, X_{t+k})|\), which is at most \(\sqrt{\mathrm{Var}X_{t}\,\mathrm{Var}X_{t+k}} = \gamma(0)\). So the true inequality is the reverse, \(|\gamma(k)| \le \gamma(0)\).

The others are standard properties of an autocovariance function: \(\gamma(0)\) is the variance (A), \(\gamma\) is even (B), and \(\gamma\) is non-negative definite (D), because \(\sum\sum \alpha_{i}\alpha_{j}\gamma(|t_{i} - t_{j}|)\) equals \(\mathrm{Var}(\sum \alpha_{i}X_{t_{i}}) \ge 0\).

Answer: C

The true statement is that \(\{Y_{t}\}\) is causal, but not invertible.

\(Y_{t}\) is ARMA(1, 1) with AR polynomial \(1 - 0.4z\), whose root \(z = 2.5\) lies outside the unit circle, so it is causal. Its MA polynomial \(1 + 1.2z\) has root \(z = -1/1.2 \approx -0.83\), inside the unit circle, so it is not invertible.

\(X_{t}\) is AR(2) with polynomial \(1 - 0.3z + 0.2z^{2}\). Its roots are complex with \(|z|^{2} = 1/0.2 = 5\), so \(|z| = \sqrt{5} \gt 1\) and \(X_{t}\) is causal; a pure AR process is always invertible. So B and D are false, and A has Y's properties reversed.

Answer: A

The true statements are (ii) and (iii).

Taking expectations of \(X_{t} = \mu + \phi X_{t-1} + Z_{t}\) under stationarity gives \(E(X_{t}) = \mu/(1 - \phi)\), not \(\mu\), so (i) is false. Taking variances gives \(\gamma(0) = \phi^{2}\gamma(0) + \sigma^{2}\), so \(V(X_{t}) = \sigma^{2}/(1 - \phi^{2})\), statement (ii) as printed without brackets. The autocorrelation of an AR(1) at lag \(s\) is \(\phi^{s}\) (iii).

(iv) leaves out \(\sigma^{2}\). So the options including (i) or (iv) are wrong.