Markov Chains and Stochastic Processes Questions and Answers

Q: Consider a Markov chain on \(S = \{1, 2, 3, 4, 5\}\) with its transition probability matrix given by :P1234513/401/40021/2001/2031/201/20040002/31/350001/32/3Then :

There exist infinitely many stationary distributions.From states 1 and 3 the chain moves only between 1 and 3, so {1, 3} is a closed recurrent class. Likewise {4, 5} is closed and recurrent. State 2 leads to 1 or 4 and never returns, so it is transient.Each closed class has its own stationary distribution: on {1, 3}, \(\pi_{1} = 2/3, \pi_{3} = 1/3\); on {4, 5}, \(\pi_{4} = \pi_{5} = 1/2\). Every convex combination of the two is also stationary, giving infinitely many.Uniqueness (A) needs a single recurrent class. Not all states are transient (C), and with a finite chain at least one stationary distribution always exists (B fails).

Q: Let \(\{X(t), t \ge 0\}\) be a time-homogeneous Poisson process with rate \(\lambda\). Let T be a positive continuous r.v. distributed independently of \(\{X(t), t \ge 0\}\). Then :

\(\mathrm{Cov}(X(T), T) = \) \(\lambda\,\mathrm{Var}(T)\).Condition on T: given \(T = t\), \(X(t)\) is Poisson with mean \(\lambda t\), so \(E[X(T) \mid T] = \lambda T\). Then\[ \begin{aligned} \mathrm{Cov}(X(T), T) &= E[T\,E(X(T) \mid T)] - E[X(T)]E[T] \\ &= \lambda E[T^{2}] - \lambda (E[T])^{2} = \lambda\,\mathrm{Var}(T) \end{aligned} \]The covariance is non-negative, so D is wrong; it depends on the distribution of T, so a constant 1/2 is wrong; and \(\lambda/(\lambda + E(T))\) has no basis in this calculation.

Q: Let the off-spring probability distributions of the three Branching processes be given by :I. \(p_{0} = \frac{1}{4}\), \(p_{1} = \frac{1}{4}\), \(p_{2} = \frac{1}{2}\)II. \(p_{0} = \frac{1}{4}\), \(p_{1} = \frac{1}{2}\), \(p_{2} = \frac{1}{4}\)III. \(p_{1} = \frac{1}{4}\), \(p_{2} = \frac{3}{4}\)Then, the extinction probabilities of the three Branching processes are respectively given by :

The extinction probabilities are 1/2, 1, 0.The extinction probability is the smallest root in [0, 1] of \(s = P(s)\), where \(P\) is the offspring generating function.I: mean \(= 1/4 + 1 = 5/4 \gt 1\). \(s = \frac{1}{4} + \frac{s}{4} + \frac{s^{2}}{2}\) gives \(2s^{2} - 3s + 1 = 0\), roots 1/2 and 1, so \(q = 1/2\).II: mean \(= 1/2 + 1/2 = 1\), with \(p_{1} \neq 1\), so extinction is certain, \(q = 1\).III: \(p_{0} = 0\), every individual has at least one child, so the population never dies out, \(q = 0\).Only option C has all three values right.

Q: If all states of a Markov chain are persistent non-null :

The chain has at least one stationary distribution.Each closed class of positive recurrent (persistent non-null) states carries a unique stationary distribution, \(\pi_{j} = 1/\mu_{j}\) with \(\mu_{j}\) the mean recurrence time. So a chain whose states are all positive recurrent has at least one stationary distribution.It need not be unique (A): with two closed classes there are infinitely many, as in Q176. B is false by the argument above. And \(\lim p_{ij}^{(n)} \gt 0\) for all \(i, j\) (C) needs irreducibility and aperiodicity; a periodic chain has no limit at all.

Q: The following are three statements related to a Markov Chain :(I) For any state \(i\), \(\mu i \ge fii\)(II) For a persistent state \(i\), \(\mu i \ge 1\)(III) For a transient state \(i\), \(\mu i \lt 1\)where \(\mu i\) denote the mean recurrence time for state \(i\) and \(fii\) is the probability of first return to state \(i\).Which of the following is true ?

Both (I) and (II) are true.Write \(f_{ii}^{(n)}\) for the probability of first return at step \(n\), so \(f_{ii} = \sum_{n} f_{ii}^{(n)}\) and the mean recurrence time is \(\mu_{i} = \sum_{n} n f_{ii}^{(n)}\) (taken as \(\infty\) for a transient state).(I) Since \(n \ge 1\) in every term, \(\mu_{i} = \sum n f_{ii}^{(n)} \ge \sum f_{ii}^{(n)} = f_{ii}\). (II) For a persistent state \(f_{ii} = 1\), so (I) gives \(\mu_{i} \ge 1\); a return takes at least one step.(III) is false: for a transient state the chain may never return, so the mean recurrence time is infinite, not less than 1. That rules out the options containing (III) and the option 'only (I)'.

Q: Suppose \(f_{ij}\) denotes the probability of the first visit to \(j\) from \(i\) in a Markov chain with finite state space. The following are two statements :(I) If C is a closed communicating class of persistent states, then for any transient state \(i\) which leads to \(j, k \in C\), \(f_{ij} = f_{ik}\).(II) If C is a single closed communicating class and if \(i \to j\) where \(i \notin C\) and \(j \in C\), then \(f_{ij} = 1\).Which of the following is true ?

Both (I) and (II) are true.(I) Inside a closed communicating class C of persistent states, once the chain enters C it stays there and visits every state of C with probability 1. So starting from a transient \(i\), the chain reaches \(j \in C\) exactly when it enters C, and the same holds for \(k \in C\). Both \(f_{ij}\) and \(f_{ik}\) equal the probability of absorption into C.(II) In a finite chain, the states outside the closed classes are transient and are left for good with probability 1. If C is the only closed class, the chain must eventually enter C, and then it hits every state of C. So \(f_{ij} = 1\) for \(i \notin C\), \(j \in C\).Finiteness matters for (II): with infinitely many transient states the chain could wander off and never reach C. As stated, both hold, so the other options are wrong.

Q: Suppose \(f_{ii}^{(n)}\) denotes the probability of the first return to \(i\) in \(n\) steps in a Markov chain. The following are three statements :(I) For any state \(i\), \(f_{ii}^{(n)} \le p_{ii}^{(n)}\), \(n \ge 1\).(II) For any state \(i\), \(\sum_{n \ge 1} p_{ii}^{(n)} \ge 1\).(III) For a persistent state \(i\), \(\Sigma p_{ii}^{(n)} \ge 1\).Which of the following is true ?

Both (I) and (III) are true.(I) The event 'first return to \(i\) at step \(n\)' is contained in the event 'at \(i\) at step \(n\)', so \(f_{ii}^{(n)} \le p_{ii}^{(n)}\) for every \(n \ge 1\).(III) A state is persistent exactly when \(\sum_{n} p_{ii}^{(n)} = \infty\), so the sum is certainly at least 1.(II) fails for transient states. If \(i\) can be left and never revisited, all \(p_{ii}^{(n)}\) are 0 and the sum is 0. More generally \(\sum p_{ii}^{(n)} = f_{ii}/(1 - f_{ii})\), which is below 1 whenever \(f_{ii} \lt 1/2\). So 'all three' and 'both (I) and (II)' are wrong, and 'only (I)' misses (III).

Q: The following are four statements about a Markov chain. A unique stationary distribution exists if the Markov chain is irreducible and (I) ergodic (II) non-null persistent, (III) transient and (IV) null persistent.Which of the following is true ?

Both (I) and (II) are true.An irreducible Markov chain has a stationary distribution if and only if it is positive (non-null) recurrent, and then the stationary distribution is unique, with \(\pi_{j} = 1/\mu_{j}\), the reciprocal of the mean recurrence time. So (II) is sufficient. An ergodic chain is aperiodic and non-null persistent, so (I) is also sufficient, and in that case \(p_{ij}^{(n)} \to \pi_{j}\) as well.For transient or null persistent irreducible chains, \(p_{ij}^{(n)} \to 0\) and no stationary distribution exists (a stationary measure, if any, cannot be normalised). So (III) and (IV) fail, and the option 'only (I)' or 'only (II)' leaves out the other correct condition.

Q: Which of the following is true ?A linear death process is a continuous time Markov chain, where :

In a linear death process, 0 is a non-null persistent state and all other states are transient.In a linear death process the population of size \(n\) falls to \(n - 1\) at rate \(n\mu\), and there are no births. State 0 is absorbing: once the population dies out it stays at 0 forever. An absorbing state is persistent with mean recurrence time finite (the chain never leaves), so it is non-null persistent.Every state \(n \ge 1\) is left with certainty and can never be re-entered, because the population only decreases. So every such state is transient. That rules out 'all states non-null persistent', 'all states transient' (0 is not), and 'other states null persistent'.

21.

Consider a Markov chain on \(S = \{1, 2, 3, 4, 5\}\) with its transition probability matrix given by :

P	1	3	4	5
1	3/4	1/4	0	0
2	1/2	0	1/2	0
3	1/2	1/2	0	0
4	0	0	2/3	1/3
5	0	0	1/3	2/3

Then :

A.there exists a unique stationary distribution
B.there does not exist any stationary distribution
C.all states are transient
D.there exist infinitely many stationary distributions

Exam Year(s)

MH SET - 2021, SET Exam - 2021,

Answer: D

There exist infinitely many stationary distributions.

From states 1 and 3 the chain moves only between 1 and 3, so {1, 3} is a closed recurrent class. Likewise {4, 5} is closed and recurrent. State 2 leads to 1 or 4 and never returns, so it is transient.

Each closed class has its own stationary distribution: on {1, 3}, \(\pi_{1} = 2/3, \pi_{3} = 1/3\); on {4, 5}, \(\pi_{4} = \pi_{5} = 1/2\). Every convex combination of the two is also stationary, giving infinitely many.

Uniqueness (A) needs a single recurrent class. Not all states are transient (C), and with a finite chain at least one stationary distribution always exists (B fails).

Discuss in Forum

22.

Let \(\{X(t), t \ge 0\}\) be a time-homogeneous Poisson process with rate \(\lambda\). Let T be a positive continuous r.v. distributed independently of \(\{X(t), t \ge 0\}\). Then :

A.\(\mathrm{Cov}(X(T), T) = \dfrac{1}{2}\)
B.\(\mathrm{Cov}(X(T), T) = \lambda/(\lambda + E(T))\)
C.\(\mathrm{Cov}(X(T), T) = \lambda\,\mathrm{Var}(T)\)
D.\(\mathrm{Cov}(X(T), T) \lt 0\)

Exam Year(s)

MH SET - 2021, SET Exam - 2021,

Answer: C

\(\mathrm{Cov}(X(T), T) = \) \(\lambda\,\mathrm{Var}(T)\).

Condition on T: given \(T = t\), \(X(t)\) is Poisson with mean \(\lambda t\), so \(E[X(T) \mid T] = \lambda T\). Then

\[ \begin{aligned} \mathrm{Cov}(X(T), T) &= E[T\,E(X(T) \mid T)] - E[X(T)]E[T] \\ &= \lambda E[T^{2}] - \lambda (E[T])^{2} = \lambda\,\mathrm{Var}(T) \end{aligned} \]

The covariance is non-negative, so D is wrong; it depends on the distribution of T, so a constant 1/2 is wrong; and \(\lambda/(\lambda + E(T))\) has no basis in this calculation.

Discuss in Forum

23.

Let the off-spring probability distributions of the three Branching processes be given by :

I. \(p_{0} = \frac{1}{4}\), \(p_{1} = \frac{1}{4}\), \(p_{2} = \frac{1}{2}\)
II. \(p_{0} = \frac{1}{4}\), \(p_{1} = \frac{1}{2}\), \(p_{2} = \frac{1}{4}\)
III. \(p_{1} = \frac{1}{4}\), \(p_{2} = \frac{3}{4}\)

Then, the extinction probabilities of the three Branching processes are respectively given by :

A.0, 0, 1
B.1/2, 0, 1
C.1/2, 1, 0
D.1/2, 1/2, 1/2

Exam Year(s)

MH SET - 2021, SET Exam - 2021,

Answer: C

The extinction probabilities are 1/2, 1, 0.

The extinction probability is the smallest root in [0, 1] of \(s = P(s)\), where \(P\) is the offspring generating function.

I: mean \(= 1/4 + 1 = 5/4 \gt 1\). \(s = \frac{1}{4} + \frac{s}{4} + \frac{s^{2}}{2}\) gives \(2s^{2} - 3s + 1 = 0\), roots 1/2 and 1, so \(q = 1/2\).
II: mean \(= 1/2 + 1/2 = 1\), with \(p_{1} \neq 1\), so extinction is certain, \(q = 1\).
III: \(p_{0} = 0\), every individual has at least one child, so the population never dies out, \(q = 0\).

Only option C has all three values right.

Discuss in Forum

24.

If all states of a Markov chain are persistent non-null :

A.it has a unique stationary distribution
B.it does not have a stationary distribution
C.\(\lim_{n \to \infty} p_{ij}^{(x)} \gt 0\) for all \(i\) and \(j\)
D.it has at least one stationary distribution

Exam Year(s)

MH SET - 2021, SET Exam - 2021,

Answer: D

The chain has at least one stationary distribution.

Each closed class of positive recurrent (persistent non-null) states carries a unique stationary distribution, \(\pi_{j} = 1/\mu_{j}\) with \(\mu_{j}\) the mean recurrence time. So a chain whose states are all positive recurrent has at least one stationary distribution.

It need not be unique (A): with two closed classes there are infinitely many, as in Q176. B is false by the argument above. And \(\lim p_{ij}^{(n)} \gt 0\) for all \(i, j\) (C) needs irreducibility and aperiodicity; a periodic chain has no limit at all.

Discuss in Forum

25.

The following are three statements related to a Markov Chain :

(I) For any state \(i\), \(\mu i \ge fii\)
(II) For a persistent state \(i\), \(\mu i \ge 1\)
(III) For a transient state \(i\), \(\mu i \lt 1\)

where \(\mu i\) denote the mean recurrence time for state \(i\) and \(fii\) is the probability of first return to state \(i\).

Which of the following is true ?

A.Only (I) is true
B.Both (I) and (II) are true
C.Both (I) and (III) are true
D.Only (III) is true

Exam Year(s)

MH SET - 2023, SET Exam - 2023,

Answer: B

Both (I) and (II) are true.

Write \(f_{ii}^{(n)}\) for the probability of first return at step \(n\), so \(f_{ii} = \sum_{n} f_{ii}^{(n)}\) and the mean recurrence time is \(\mu_{i} = \sum_{n} n f_{ii}^{(n)}\) (taken as \(\infty\) for a transient state).

(I) Since \(n \ge 1\) in every term, \(\mu_{i} = \sum n f_{ii}^{(n)} \ge \sum f_{ii}^{(n)} = f_{ii}\). (II) For a persistent state \(f_{ii} = 1\), so (I) gives \(\mu_{i} \ge 1\); a return takes at least one step.

(III) is false: for a transient state the chain may never return, so the mean recurrence time is infinite, not less than 1. That rules out the options containing (III) and the option 'only (I)'.

Discuss in Forum

26.

Suppose \(f_{ij}\) denotes the probability of the first visit to \(j\) from \(i\) in a Markov chain with finite state space. The following are two statements :

(I) If C is a closed communicating class of persistent states, then for any transient state \(i\) which leads to \(j, k \in C\), \(f_{ij} = f_{ik}\).
(II) If C is a single closed communicating class and if \(i \to j\) where \(i \notin C\) and \(j \in C\), then \(f_{ij} = 1\).

Which of the following is true ?

A.Both (I) and (II) are false
B.Both (I) and (II) are true
C.(I) is true but (II) is false
D.(I) is false but (II) is true

Exam Year(s)

MH SET - 2023, SET Exam - 2023,

Answer: B

Both (I) and (II) are true.

(I) Inside a closed communicating class C of persistent states, once the chain enters C it stays there and visits every state of C with probability 1. So starting from a transient \(i\), the chain reaches \(j \in C\) exactly when it enters C, and the same holds for \(k \in C\). Both \(f_{ij}\) and \(f_{ik}\) equal the probability of absorption into C.

(II) In a finite chain, the states outside the closed classes are transient and are left for good with probability 1. If C is the only closed class, the chain must eventually enter C, and then it hits every state of C. So \(f_{ij} = 1\) for \(i \notin C\), \(j \in C\).

Finiteness matters for (II): with infinitely many transient states the chain could wander off and never reach C. As stated, both hold, so the other options are wrong.

Discuss in Forum

27.

Suppose \(f_{ii}^{(n)}\) denotes the probability of the first return to \(i\) in \(n\) steps in a Markov chain. The following are three statements :

(I) For any state \(i\), \(f_{ii}^{(n)} \le p_{ii}^{(n)}\), \(n \ge 1\).
(II) For any state \(i\), \(\sum_{n \ge 1} p_{ii}^{(n)} \ge 1\).
(III) For a persistent state \(i\), \(\Sigma p_{ii}^{(n)} \ge 1\).

Which of the following is true ?

A.All three are true
B.Both (I) and (II) are true
C.Both (I) and (III) are true
D.Only (I) is true

Exam Year(s)

MH SET - 2023, SET Exam - 2023,

Answer: C

Both (I) and (III) are true.

(I) The event 'first return to \(i\) at step \(n\)' is contained in the event 'at \(i\) at step \(n\)', so \(f_{ii}^{(n)} \le p_{ii}^{(n)}\) for every \(n \ge 1\).

(III) A state is persistent exactly when \(\sum_{n} p_{ii}^{(n)} = \infty\), so the sum is certainly at least 1.

(II) fails for transient states. If \(i\) can be left and never revisited, all \(p_{ii}^{(n)}\) are 0 and the sum is 0. More generally \(\sum p_{ii}^{(n)} = f_{ii}/(1 - f_{ii})\), which is below 1 whenever \(f_{ii} \lt 1/2\). So 'all three' and 'both (I) and (II)' are wrong, and 'only (I)' misses (III).

Discuss in Forum

28.

The following are four statements about a Markov chain. A unique stationary distribution exists if the Markov chain is irreducible and (I) ergodic (II) non-null persistent, (III) transient and (IV) null persistent.

Which of the following is true ?

A.Only (I) is true
B.Only (II) is true
C.Both (I) and (II) are true
D.Either (III) or (IV) is true

Exam Year(s)

MH SET - 2023, SET Exam - 2023,

Answer: C

Both (I) and (II) are true.

An irreducible Markov chain has a stationary distribution if and only if it is positive (non-null) recurrent, and then the stationary distribution is unique, with \(\pi_{j} = 1/\mu_{j}\), the reciprocal of the mean recurrence time. So (II) is sufficient. An ergodic chain is aperiodic and non-null persistent, so (I) is also sufficient, and in that case \(p_{ij}^{(n)} \to \pi_{j}\) as well.

For transient or null persistent irreducible chains, \(p_{ij}^{(n)} \to 0\) and no stationary distribution exists (a stationary measure, if any, cannot be normalised). So (III) and (IV) fail, and the option 'only (I)' or 'only (II)' leaves out the other correct condition.

Discuss in Forum

29.

Which of the following is true ?

The life time of a component of a machine is modeled by the exponential distribution with mean 2 per week. Failed component is immediately replaced by a new one. Then the probability that no component is replaced in two weeks is :

A.\(e^{-2}\)
B.\(e^{-4}\)
C.\(e^{-0.5}\)
D.\(e^{-1}\)

Exam Year(s)

MH SET - 2023, SET Exam - 2023,

Answer: D

The probability is \(e^{-1}\), reading the exponential lifetime as having mean 2 weeks.

No replacement in two weeks means the component in use at the start lasts more than 2 weeks. With mean lifetime 2 weeks, the failure rate is \(\lambda = 1/2\) per week, and

\[ \begin{aligned} P(T \gt 2) &= e^{-\lambda t} = e^{-(1/2)(2)} \\ &= e^{-1} \approx 0.368 \end{aligned} \]

The memoryless property makes this hold even if the component was not new at time 0. The phrase 'mean 2 per week' in the paper is loose; if 2 were the failure rate per week, the answer would be \(e^{-4}\), but the key takes 2 as the mean life in weeks. The values \(e^{-2}\) and \(e^{-0.5}\) correspond to other pairings of rate and time (rate 1 for two weeks, rate 1/2 for one week).

Discuss in Forum

30.

Which of the following is true ?

A linear death process is a continuous time Markov chain, where :

A.all states are non-null persistent
B.all states are transient
C.0 is a non-null persistent state and all other states are transient
D.0 is a non-null persistent state and all other states are null persistent

Exam Year(s)

MH SET - 2023, SET Exam - 2023,

Answer: C

In a linear death process, 0 is a non-null persistent state and all other states are transient.

In a linear death process the population of size \(n\) falls to \(n - 1\) at rate \(n\mu\), and there are no births. State 0 is absorbing: once the population dies out it stays at 0 forever. An absorbing state is persistent with mean recurrence time finite (the chain never leaves), so it is non-null persistent.

Every state \(n \ge 1\) is left with certainty and can never be re-entered, because the population only decreases. So every such state is transient. That rules out 'all states non-null persistent', 'all states transient' (0 is not), and 'other states null persistent'.

Markov Chains and Stochastic Processes Questions and Answers

Topics

Other Subjects

Tags: Markov Chains and Stochastic Processes Questions and Answers

About Us

Markov Chains and Stochastic Processes Questions and Answers

Topics

Other Subjects

Tags: Markov Chains and Stochastic Processes Questions and Answers

About Us

Stay informed about all important Dates & Notifications