Markov Chains and Stochastic Processes Questions and Answers

Q: Consider a discrete time Markov chain with state space \(S = \{1, 2\}\) and the transition probability diagramFromTo 1To 210.250.7520.50.5If \(\pi = (\pi_{1}, \pi_{2})\) is the stationary distribution of the Markov chain, then which one of the following statements is true?

The stationary distribution is \((\pi_{1}, \pi_{2}) = \left(\frac{2}{5}, \frac{3}{5}\right)\).For a two-state chain the stationary distribution balances the flow between the states: the probability flowing from 1 to 2 equals that flowing from 2 to 1.\[ \begin{aligned} 0.75\,\pi_{1} &= 0.5\,\pi_{2} \Rightarrow \pi_{2} = 1.5\,\pi_{1} \\ \pi_{1} + 1.5\,\pi_{1} &= 1 \Rightarrow \pi_{1} = \frac{2}{5} \end{aligned} \]so \(\pi_{2} = \frac{3}{5}\). Check with the first column: \(0.25 \times \frac{2}{5} + 0.5 \times \frac{3}{5}\) \(= 0.1 + 0.3 = 0.4 = \pi_{1}\).\(\left(\frac{1}{2}, \frac{1}{2}\right)\) would need equal flows 0.75 and 0.5. \(\left(\frac{1}{4}, \frac{3}{4}\right)\) simply copies the first row, and \(\left(\frac{1}{3}, \frac{2}{3}\right)\) fails the balance equation (\(0.25 \ne 0.33\)).

Q: Consider a discrete time Markov chain with state space \(S = \{1, 2, 3\}\) and the transition probability matrix\[ \begin{bmatrix} \frac{1}{2} & \frac{1}{4} & \frac{1}{4} \\ 0 & \frac{1}{5} & \frac{4}{5} \\ 0 & 1 & 0 \end{bmatrix}. \]Which one of the following statements is true?

The true statement is 3 is a recurrent state.Read the matrix row by row. From state 1 the chain can move to 2 or 3, but the first column is zero below the top entry, so no state leads back to 1. From 2 the chain goes to 2 or 3, and from 3 it always goes to 2.So \(\{2, 3\}\) is a closed communicating class: once there, the chain never leaves, and 2 and 3 reach each other. A finite closed communicating class is recurrent, so both 2 and 3 are recurrent. State 1 is left with probability \(\frac{1}{2}\) at each step and never re-entered, so it is transient.That rules out the other options: the chain is not irreducible, since 1 cannot be reached from 2 or 3; 1 is transient, not recurrent; and 2 is recurrent, not transient.

Q: Let \(\{(X_{n}, F_{n})\}_{n=1}^{\infty}\) be a martingale. Let \(\zeta_{n} = \sigma(X_{1}, X_{2}, ......, X_{n})\). Then, \(\{(X_{n}, \zeta_{n})\}_{n=1}^{\infty}\) is a ................ .

\(\{(X_{n}, \zeta_{n})\}\) is a martingale.Since \(X_{n}\) is \(F_{n}\)-measurable for each \(n\), the natural filtration satisfies \(\zeta_{n} = \sigma(X_{1}, \ldots, X_{n}) \subseteq F_{n}\). By the tower property,\[ \begin{aligned} E(X_{n+1}|\zeta_{n}) &= E(E(X_{n+1}|F_{n})|\zeta_{n}) \\ &= E(X_{n}|\zeta_{n}) = X_{n} \end{aligned} \]Integrability is unchanged, so the process is a martingale with respect to its own filtration. Sub- and supermartingales would need an inequality, which does not arise here, and white noise is an uncorrelated sequence, not this structure.

Q: Consider the time series model :\[ X_{t} = 0.2X_{t-2} - 0.6X_{t-1} + Z_{t} + 1.2Z_{t-1}, \]where \(Z_{t} \sim\) iid Normal \((0, \sigma^{2})\). Which of the following statements is true ?

The process is causal but not invertible.Write the model as \(\phi(B)X_{t} = \theta(B)Z_{t}\) with \(\phi(z) = 1 + 0.6z - 0.2z^{2}\) and \(\theta(z) = 1 + 1.2z\). Causality needs the roots of \(\phi\) outside the unit circle: \(0.2z^{2} - 0.6z - 1 = 0\) gives \(z^{2} - 3z - 5 = 0\), so\[ z = \frac{3 \pm \sqrt{29}}{2} \approx 4.19,\ -1.19 \]both of modulus greater than 1: causal (and stationary). Invertibility needs the root of \(\theta\) outside the unit circle, but \(1 + 1.2z = 0\) gives \(z = -0.83\), inside. So the process is not invertible, which rules out A, B and D. (The equivalent invertible representation would use the MA coefficient \(1/1.2\).)

Q: The sample autocorrelation of certain time series data was found to be significant at lags one and five, whereas the sample partial autocorrelations were not significants for first 30 lags. What model would you suggest for such a time series ?

The suggested model is MA(5).The signature of a moving average process of order \(q\) is an autocorrelation function that cuts off after lag \(q\), with partial autocorrelations that tail off. Significant autocorrelations at lags 1 and 5 only, with none of the partial autocorrelations significant, point to an MA model of order 5, with the coefficients at lags 2 to 4 possibly zero.An AR(5) model would show the reverse pattern: partial autocorrelations cutting off after lag 5. The mixed ARMA models would show both functions tailing off, and significant partial autocorrelations would be expected.

Q: In an AR(1) model \(X_{t} = 0.5X_{t-1} + Z_{t}\), \(Z_{t} \sim\) iid normal (0, 1), the best linear predictor \(aX_{1} + bX_{3}\) of \(X_{2}\) using \((X_{1}, X_{3})\) will have :

The coefficients satisfy \(a = b\).For AR(1) with \(\phi = 0.5\), the autocorrelations are \(\rho(1) = 0.5\) and \(\rho(2) = 0.25\). The best linear predictor of \(X_{2}\) from \(X_{1}, X_{3}\) solves\[ \begin{aligned} a + \rho(2)b &= \rho(1) \\ \rho(2)a + b &= \rho(1) \end{aligned} \]The system is symmetric in \(a\) and \(b\), so \(a = b = \rho(1)/(1 + \rho(2)) = 0.5/1.25 = 0.4\), which is the general \(\phi/(1 + \phi^{2})\). The reason is time reversibility: a stationary Gaussian AR(1) looks the same run backwards, so the past and future neighbours of \(X_{2}\) carry equal weight. It can be determined, so D is wrong.

Q: Let the time series \(\{Y_{t}\}\) be an ARIMA(2, 1, 2). Then \(\{Y_{t}\}\) is :

An ARIMA(2, 1, 2) process is having one unit root for the AR polynomial.ARIMA(\(p, d, q\)) means that the \(d\)th difference is a stationary, invertible ARMA(\(p, q\)). So \((1 - B)Y_{t}\) is ARMA(2, 2), and the full autoregressive operator is \(\phi(B)(1 - B)\), of degree 3, with exactly one root at \(z = 1\) coming from the single difference.Because of that unit root, \(Y_{t}\) itself is not stationary (A). The MA polynomial of an ARIMA model is taken to be invertible, with all roots outside the unit circle, so it has no unit root (C and D).

Q: Consider the MC consisting of the three states 0, 1, 2 and having TPM :\[ P = \begin{pmatrix} \frac{1}{2} & \frac{1}{2} & 0 \\ \frac{1}{2} & \frac{1}{4} & \frac{1}{4} \\ 0 & \frac{1}{3} & \frac{2}{3} \end{pmatrix} \]Which of the following is correct ?

The correct statement is MC is irreducible.From the matrix, state 0 goes to 1 (\(p_{01} = 1/2\)), 1 goes to 0 and to 2 (\(p_{10} = 1/2\), \(p_{12} = 1/4\)), and 2 goes to 1 (\(p_{21} = 1/3\)). So every state can reach every other through state 1: all states communicate and the chain is irreducible, which rules out A.A finite irreducible chain has a unique stationary distribution, so B is false. And\[ \begin{aligned} P_{00}^{(2)} &= \tfrac{1}{2} \cdot \tfrac{1}{2} + \tfrac{1}{2} \cdot \tfrac{1}{2} + 0 \\ &= \tfrac{1}{2} \end{aligned} \]not 1/4, so C is false.

Q: Let \(\{X_{t}\}\) be a time series defined as \(X_{t} = A\sin(\omega t + B)\), where \(E(A) = 0\), \(\mathrm{Var}(A) = 1\), \(B \sim\) Uniform \((-\pi, \pi)\) and A and B are independent. Then, \(h\)-lag covariance function \(r(h)\) is :

\(r(h) = \) \(\tfrac{1}{2}\cos(\omega h)\).Since \(E(A) = 0\), the mean is 0, and by independence\[ \begin{aligned} r(h) &= E(A^{2}) \\ &\quad \times E[\sin(\omega t + B)\sin(\omega t + \omega h + B)] \end{aligned} \]with \(E(A^{2}) = 1\). Using \(\sin u \sin v = \tfrac{1}{2}[\cos(u - v) - \cos(u + v)]\), the first term is \(\tfrac{1}{2}\cos(\omega h)\) and the second, \(\tfrac{1}{2}E\cos(2\omega t + \omega h + 2B)\), averages to 0 over B uniform on \((-\pi, \pi)\).So \(r(h) = \tfrac{1}{2}\cos\omega h\), which depends only on the lag, confirming stationarity. An autocovariance must be even in \(h\) and equal to the variance at \(h = 0\), which rules out the sine forms and the shifted cosine.

Q: Consider a Markov chain on S = {1, 2, 3, 4} with the transition probability matrix given by :1234100012000131/21/20040010Then :

The limits do not exist, and there is a unique stationary distribution.The moves are \(1 \to 4\), \(2 \to 4\), \(4 \to 3\) and \(3 \to 1\) or 2. Every state reaches every other, so the chain is irreducible, and every return path has length 3 (\(4 \to 3 \to 1 \to 4\) or \(4 \to 3 \to 2 \to 4\)), so it has period 3.A finite irreducible chain has a unique stationary distribution. Solving \(\pi = \pi P\): \(\pi_{3} = \pi_{4} = \pi_{1} + \pi_{2}\) and \(\pi_{1} = \pi_{2} = \pi_{3}/2\), so \(\pi = (1/6, 1/6, 1/3, 1/3)\).Because of the period, \(p_{ii}^{(n)}\) is 0 unless 3 divides \(n\), so the n-step probabilities oscillate and have no limit. That rules out A, and C and D are wrong since there is exactly one stationary distribution.

Consider a discrete time Markov chain with state space \(S = \{1, 2\}\) and the transition probability diagram

From	To 1	To 2
1	0.25	0.75
2	0.5	0.5

If \(\pi = (\pi_{1}, \pi_{2})\) is the stationary distribution of the Markov chain, then which one of the following statements is true?

A.\((\pi_{1}, \pi_{2}) = \left(\frac{1}{2}, \frac{1}{2}\right)\)
B.\((\pi_{1}, \pi_{2}) = \left(\frac{2}{5}, \frac{3}{5}\right)\)
C.\((\pi_{1}, \pi_{2}) = \left(\frac{1}{4}, \frac{3}{4}\right)\)
D.\((\pi_{1}, \pi_{2}) = \left(\frac{1}{3}, \frac{2}{3}\right)\)

Exam Year(s)

IIT JAM - 2026,

Answer: B

The stationary distribution is \((\pi_{1}, \pi_{2}) = \left(\frac{2}{5}, \frac{3}{5}\right)\).

For a two-state chain the stationary distribution balances the flow between the states: the probability flowing from 1 to 2 equals that flowing from 2 to 1.

\[ \begin{aligned} 0.75\,\pi_{1} &= 0.5\,\pi_{2} \Rightarrow \pi_{2} = 1.5\,\pi_{1} \\ \pi_{1} + 1.5\,\pi_{1} &= 1 \Rightarrow \pi_{1} = \frac{2}{5} \end{aligned} \]

so \(\pi_{2} = \frac{3}{5}\). Check with the first column: \(0.25 \times \frac{2}{5} + 0.5 \times \frac{3}{5}\) \(= 0.1 + 0.3 = 0.4 = \pi_{1}\).

\(\left(\frac{1}{2}, \frac{1}{2}\right)\) would need equal flows 0.75 and 0.5. \(\left(\frac{1}{4}, \frac{3}{4}\right)\) simply copies the first row, and \(\left(\frac{1}{3}, \frac{2}{3}\right)\) fails the balance equation (\(0.25 \ne 0.33\)).

Discuss in Forum

Consider a discrete time Markov chain with state space \(S = \{1, 2, 3\}\) and the transition probability matrix

\[ \begin{bmatrix} \frac{1}{2} & \frac{1}{4} & \frac{1}{4} \\ 0 & \frac{1}{5} & \frac{4}{5} \\ 0 & 1 & 0 \end{bmatrix}. \]

Which one of the following statements is true?

A.The Markov chain is irreducible
B.\(1\) is a recurrent state
C.\(2\) is a transient state
D.\(3\) is a recurrent state

Exam Year(s)

IIT JAM - 2026,

Answer: D

The true statement is 3 is a recurrent state.

Read the matrix row by row. From state 1 the chain can move to 2 or 3, but the first column is zero below the top entry, so no state leads back to 1. From 2 the chain goes to 2 or 3, and from 3 it always goes to 2.

So \(\{2, 3\}\) is a closed communicating class: once there, the chain never leaves, and 2 and 3 reach each other. A finite closed communicating class is recurrent, so both 2 and 3 are recurrent. State 1 is left with probability \(\frac{1}{2}\) at each step and never re-entered, so it is transient.

That rules out the other options: the chain is not irreducible, since 1 cannot be reached from 2 or 3; 1 is transient, not recurrent; and 2 is recurrent, not transient.

Discuss in Forum

Let \(\{(X_{n}, F_{n})\}_{n=1}^{\infty}\) be a martingale. Let \(\zeta_{n} = \sigma(X_{1}, X_{2}, ......, X_{n})\). Then, \(\{(X_{n}, \zeta_{n})\}_{n=1}^{\infty}\) is a ................ .

A.Martingale
B.Sub Martingale
C.Super Martingale
D.White noise

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: A

\(\{(X_{n}, \zeta_{n})\}\) is a martingale.

Since \(X_{n}\) is \(F_{n}\)-measurable for each \(n\), the natural filtration satisfies \(\zeta_{n} = \sigma(X_{1}, \ldots, X_{n}) \subseteq F_{n}\). By the tower property,

\[ \begin{aligned} E(X_{n+1}|\zeta_{n}) &= E(E(X_{n+1}|F_{n})|\zeta_{n}) \\ &= E(X_{n}|\zeta_{n}) = X_{n} \end{aligned} \]

Integrability is unchanged, so the process is a martingale with respect to its own filtration. Sub- and supermartingales would need an inequality, which does not arise here, and white noise is an uncorrelated sequence, not this structure.

Discuss in Forum

Consider the time series model :

\[ X_{t} = 0.2X_{t-2} - 0.6X_{t-1} + Z_{t} + 1.2Z_{t-1}, \]

where \(Z_{t} \sim\) iid Normal \((0, \sigma^{2})\). Which of the following statements is true ?

A.\(\{X_{t}\}\) is stationary and invertible
B.\(\{X_{t}\}\) is invertible, but not causal
C.\(\{X_{t}\}\) is causal but not invertible
D.\(\{X_{t}\}\) is neither causal nor invertible

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: C

The process is causal but not invertible.

Write the model as \(\phi(B)X_{t} = \theta(B)Z_{t}\) with \(\phi(z) = 1 + 0.6z - 0.2z^{2}\) and \(\theta(z) = 1 + 1.2z\). Causality needs the roots of \(\phi\) outside the unit circle: \(0.2z^{2} - 0.6z - 1 = 0\) gives \(z^{2} - 3z - 5 = 0\), so

\[ z = \frac{3 \pm \sqrt{29}}{2} \approx 4.19,\ -1.19 \]

both of modulus greater than 1: causal (and stationary). Invertibility needs the root of \(\theta\) outside the unit circle, but \(1 + 1.2z = 0\) gives \(z = -0.83\), inside. So the process is not invertible, which rules out A, B and D. (The equivalent invertible representation would use the MA coefficient \(1/1.2\).)

Discuss in Forum

The sample autocorrelation of certain time series data was found to be significant at lags one and five, whereas the sample partial autocorrelations were not significants for first 30 lags. What model would you suggest for such a time series ?

A.ARMA(1, 5)
B.ARMA(5, 1)
C.MA(5)
D.AR(5)

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: C

The suggested model is MA(5).

The signature of a moving average process of order \(q\) is an autocorrelation function that cuts off after lag \(q\), with partial autocorrelations that tail off. Significant autocorrelations at lags 1 and 5 only, with none of the partial autocorrelations significant, point to an MA model of order 5, with the coefficients at lags 2 to 4 possibly zero.

An AR(5) model would show the reverse pattern: partial autocorrelations cutting off after lag 5. The mixed ARMA models would show both functions tailing off, and significant partial autocorrelations would be expected.

Discuss in Forum

In an AR(1) model \(X_{t} = 0.5X_{t-1} + Z_{t}\), \(Z_{t} \sim\) iid normal (0, 1), the best linear predictor \(aX_{1} + bX_{3}\) of \(X_{2}\) using \((X_{1}, X_{3})\) will have :

A.\(a = b\)
B.\(a \lt b\)
C.\(a \gt b\)
D.Cannot be determined

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: A

The coefficients satisfy \(a = b\).

For AR(1) with \(\phi = 0.5\), the autocorrelations are \(\rho(1) = 0.5\) and \(\rho(2) = 0.25\). The best linear predictor of \(X_{2}\) from \(X_{1}, X_{3}\) solves

\[ \begin{aligned} a + \rho(2)b &= \rho(1) \\ \rho(2)a + b &= \rho(1) \end{aligned} \]

The system is symmetric in \(a\) and \(b\), so \(a = b = \rho(1)/(1 + \rho(2)) = 0.5/1.25 = 0.4\), which is the general \(\phi/(1 + \phi^{2})\). The reason is time reversibility: a stationary Gaussian AR(1) looks the same run backwards, so the past and future neighbours of \(X_{2}\) carry equal weight. It can be determined, so D is wrong.

Discuss in Forum

Let the time series \(\{Y_{t}\}\) be an ARIMA(2, 1, 2). Then \(\{Y_{t}\}\) is :

A.a stationary model
B.having one unit root for the AR polynomial
C.having one unit root for the MA polynomial
D.having unit roots for both AR and MA polynomials

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: B

An ARIMA(2, 1, 2) process is having one unit root for the AR polynomial.

ARIMA(\(p, d, q\)) means that the \(d\)th difference is a stationary, invertible ARMA(\(p, q\)). So \((1 - B)Y_{t}\) is ARMA(2, 2), and the full autoregressive operator is \(\phi(B)(1 - B)\), of degree 3, with exactly one root at \(z = 1\) coming from the single difference.

Because of that unit root, \(Y_{t}\) itself is not stationary (A). The MA polynomial of an ARIMA model is taken to be invertible, with all roots outside the unit circle, so it has no unit root (C and D).

Discuss in Forum

Consider the MC consisting of the three states 0, 1, 2 and having TPM :

\[ P = \begin{pmatrix} \frac{1}{2} & \frac{1}{2} & 0 \\ \frac{1}{2} & \frac{1}{4} & \frac{1}{4} \\ 0 & \frac{1}{3} & \frac{2}{3} \end{pmatrix} \]

Which of the following is correct ?

A.All states do not communicate
B.The stationary distribution does not exist
C.\(P_{00}^{(2)} = \dfrac{1}{4}\)
D.MC is irreducible

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: D

The correct statement is MC is irreducible.

From the matrix, state 0 goes to 1 (\(p_{01} = 1/2\)), 1 goes to 0 and to 2 (\(p_{10} = 1/2\), \(p_{12} = 1/4\)), and 2 goes to 1 (\(p_{21} = 1/3\)). So every state can reach every other through state 1: all states communicate and the chain is irreducible, which rules out A.

A finite irreducible chain has a unique stationary distribution, so B is false. And

\[ \begin{aligned} P_{00}^{(2)} &= \tfrac{1}{2} \cdot \tfrac{1}{2} + \tfrac{1}{2} \cdot \tfrac{1}{2} + 0 \\ &= \tfrac{1}{2} \end{aligned} \]

not 1/4, so C is false.

Discuss in Forum

Let \(\{X_{t}\}\) be a time series defined as \(X_{t} = A\sin(\omega t + B)\), where \(E(A) = 0\), \(\mathrm{Var}(A) = 1\), \(B \sim\) Uniform \((-\pi, \pi)\) and A and B are independent. Then, \(h\)-lag covariance function \(r(h)\) is :

A.\(\dfrac{1}{2}\cos(\omega h)\)
B.\(\dfrac{1}{2}\sin(\omega h)\)
C.\(\dfrac{1}{2}\cos(\omega h + 4)\)
D.\(\dfrac{1}{2}\sin(\omega h + 2)\)

Exam Year(s)

MH SET - 2019, SET Exam - 2019,

Answer: A

\(r(h) = \) \(\tfrac{1}{2}\cos(\omega h)\).

Since \(E(A) = 0\), the mean is 0, and by independence

\[ \begin{aligned} r(h) &= E(A^{2}) \\ &\quad \times E[\sin(\omega t + B)\sin(\omega t + \omega h + B)] \end{aligned} \]

with \(E(A^{2}) = 1\). Using \(\sin u \sin v = \tfrac{1}{2}[\cos(u - v) - \cos(u + v)]\), the first term is \(\tfrac{1}{2}\cos(\omega h)\) and the second, \(\tfrac{1}{2}E\cos(2\omega t + \omega h + 2B)\), averages to 0 over B uniform on \((-\pi, \pi)\).

So \(r(h) = \tfrac{1}{2}\cos\omega h\), which depends only on the lag, confirming stationarity. An autocovariance must be even in \(h\) and equal to the variance at \(h = 0\), which rules out the sine forms and the shifted cosine.

Discuss in Forum

10.

Consider a Markov chain on S = {1, 2, 3, 4} with the transition probability matrix given by :

	1	2	3	4
1	0	0	0	1
2	0	0	0	1
3	1/2	1/2	0	0
4	0	0	1	0

Then :

A.\(\lim_{n\to\infty} p_{ij}^{(n)}\) exists for all \((i, j)\) and the limit is independent of \(i\)
B.\(\lim_{n\to\infty} p_{ij}^{(n)}\) does not exist for all \((i, j)\) and there is a unique stationary distribution
C.there does not exist any stationary distribution
D.there exist infinitely many stationary distributions

Exam Year(s)

MH SET - 2019, SET Exam - 2019,

Answer: B

The limits do not exist, and there is a unique stationary distribution.

The moves are \(1 \to 4\), \(2 \to 4\), \(4 \to 3\) and \(3 \to 1\) or 2. Every state reaches every other, so the chain is irreducible, and every return path has length 3 (\(4 \to 3 \to 1 \to 4\) or \(4 \to 3 \to 2 \to 4\)), so it has period 3.

A finite irreducible chain has a unique stationary distribution. Solving \(\pi = \pi P\): \(\pi_{3} = \pi_{4} = \pi_{1} + \pi_{2}\) and \(\pi_{1} = \pi_{2} = \pi_{3}/2\), so \(\pi = (1/6, 1/6, 1/3, 1/3)\).

Because of the period, \(p_{ii}^{(n)}\) is 0 unless 3 divides \(n\), so the n-step probabilities oscillate and have no limit. That rules out A, and C and D are wrong since there is exactly one stationary distribution.

Markov Chains and Stochastic Processes Questions and Answers

Topics

Other Subjects

Tags: Markov Chains and Stochastic Processes Questions and Answers

About Us

Markov Chains and Stochastic Processes Questions and Answers

Topics

Other Subjects

Tags: Markov Chains and Stochastic Processes Questions and Answers

About Us

Stay informed about all important Dates & Notifications