Bayesian Inference Questions and Answers

Q: Suppose \(X_{1}\) ...... \(X_{n}\) are independent observations from a Poisson distribution with probability mass function :\[ \begin{aligned} f(x/\lambda) &= \frac{e^{-\lambda}\lambda^{x}}{x!}, \\ x &= 0, 1, 2, ......., \end{aligned} \]\(\lambda \gt 0\) unkonwn.Suppose the prior distribution of \(\lambda\) is desired as :\[ g(\lambda) \propto \lambda,\ e^{-2\lambda},\ \lambda \gt 0. \]Let \(n = 3\) and the data \((X_{1}, X_{2}, X_{3}) = (1, 1, 2)\). Then, suppose we estimate \(\lambda\) under squared error loss \(L(\lambda, a) = (\lambda - a)^{2}\). Then, the Bayes estimator based on the given data is :

The Bayes estimate is 1.2.The prior, read as \(g(\lambda) \propto \lambda e^{-2\lambda}\) (the comma in the stem stands for multiplication), is a gamma density with shape 2 and rate 2. The Poisson likelihood of the data is proportional to \(e^{-3\lambda}\lambda^{1 + 1 + 2}\). Multiplying:\[ \pi(\lambda|x) \propto \lambda^{5}e^{-5\lambda} \]a gamma posterior with shape 6 and rate 5. Under squared error loss the Bayes estimator is the posterior mean, \(6/5 = 1.2\).None of the other values is the posterior mean. Ignoring the prior would give the sample mean 4/3, and the posterior mode, \((6 - 1)/5 = 1\), is not offered either; the prior pulls the estimate below the sample mean because its own mean is 1.

Q: Let \(X_{1}, X_{2}, \ldots, X_{n}\) be i.i.d. Bernoulli \((\theta)\). Let the loss function for the problem of estimation of \(\theta\) be given by\[ L(\theta, a) = \frac{(\theta - a)^{2}}{\theta(1 - \theta)}. \]Let the pdf of the prior for \(\theta\) (\(\pi(\theta)\)) be given by \(\pi(\theta) = 1\), \(0 \le \theta \le 1\).Let \(\overline{X}\) be the sample mean. Then :

\(\bar{X}\) is both Bayes and minimax.Under the weighted loss \(L = (\theta - a)^{2}/[\theta(1 - \theta)]\), the Bayes estimator is \(E[\theta w(\theta) \mid x]/E[w(\theta) \mid x]\) with \(w = 1/[\theta(1 - \theta)]\). With the uniform prior the posterior is Beta(s + 1, n - s + 1), where \(s = \sum x_{i}\); multiplying by \(w\) gives a Beta(s, n - s) kernel, whose mean is \(s/n = \bar{X}\). So \(\bar{X}\) is Bayes.Its risk is constant:\[ \begin{aligned} \frac{E(\bar{X} - \theta)^{2}}{\theta(1 - \theta)} &= \frac{\theta(1 - \theta)/n}{\theta(1 - \theta)} \\ &= \frac{1}{n} \end{aligned} \]A Bayes estimator with constant risk is minimax, and a unique Bayes estimator is admissible, so A, B and D are wrong.

Q: Suppose we have 10 observations \(\underset{\sim}{y} = (y_{1}, y_{2}, \ldots, y_{10})\) from Poisson \((\lambda)\) and we assume a Gamma \((a, b)\) prior distribution for \(\lambda\).\[ \begin{aligned} \Big(p(\lambda) &= \{\Gamma(a)\}^{-1} b^{a} e^{-b\lambda} \lambda^{a-1}, \\ &\quad a \gt 0,\ b \gt 0\Big) \end{aligned} \]Then the posterior predictive distribution \(p(y_{\text{new}} / \underset{\sim}{y})\) of a new observation \(y_{\text{new}}\) is given by :

The posterior predictive distribution is Negative Binomial \(\left(a + \sum y_{i},\ b + 10\right)\).With a Gamma \((a, b)\) prior (rate \(b\)) and 10 Poisson observations, the posterior is Gamma \((a + \sum y_{i},\ b + 10)\), by conjugacy. A new observation is Poisson \((\lambda)\) given \(\lambda\), and mixing a Poisson over a gamma distribution gives a negative binomial:\[ \begin{aligned} &p(y_{\text{new}} \mid \underset{\sim}{y}) \\ &= \binom{\alpha + y - 1}{y} \left(\frac{\beta}{\beta + 1}\right)^{\alpha} \\ &\quad \times \left(\frac{1}{\beta + 1}\right)^{y} \end{aligned} \]with \(\alpha = a + \sum y_{i}\) and \(\beta = b + 10\), the parameters named in the option.The two gamma options describe the posterior of \(\lambda\) (one with its parameters swapped), not the distribution of a new count, which must be discrete. The Poisson option ignores the uncertainty in \(\lambda\), which is exactly what makes the predictive distribution overdispersed.

Q: Suppose\[ P(Y=y/\theta)=\binom{n}{y}\theta^{y}(1-\theta)^{n-y}, \]\(y \in \{0, 1, 2, \ldots, n\}\). If \(\theta\) ~ uniform (0, 1), then the posterior distribution will be :

The posterior is Beta(y + 1, n − y + 1).The uniform prior on (0, 1) has density 1, so the posterior is proportional to the likelihood:\[ \pi(\theta \mid y) \propto \theta^{y}(1 - \theta)^{n-y} \]This is the kernel of a Beta distribution with parameters \(y + 1\) and \(n - y + 1\) (the Beta(a, b) kernel is \(\theta^{a-1}(1 - \theta)^{b-1}\)).Beta(y, n − y) drops the 1 contributed by the uniform prior, which is Beta(1, 1). The uniform distributions ignore the data.

Q: Let X ~ Poisson (\(\lambda\)) and a single realization of X resulted in 3. If \(\lambda\) has a prior distribution\[ P(\lambda) = q\, \lambda\, e^{-3\lambda},\ \lambda \gt 0, \]then the posterior mean is given by :

The posterior mean is 5/4.The prior \(q\lambda e^{-3\lambda}\) is a Gamma density with shape 2 and rate 3. With one Poisson observation x = 3 the likelihood is proportional to \(\lambda^{3}e^{-\lambda}\), so\[ \begin{aligned} \pi(\lambda \mid x) &\propto \lambda^{3}e^{-\lambda} \cdot \lambda e^{-3\lambda} \\ &= \lambda^{4}e^{-4\lambda} \end{aligned} \]which is Gamma with shape 5 and rate 4. Its mean is \(5/4\).4/5 inverts the shape and rate, and 3/5 and 2/5 use the wrong shape or rate.

Q: Which of the following statements about the Bayes factor is true ?(I) Bayes factor involves marginal likelihood.(II) Bayes factor expresses the support given by the data for one or other of the models.(III) Bayes factor involves the conditional likelihood

The key accepts (I) and (II) as true and (III) as false.(I) The Bayes factor for models \(M_{1}\) and \(M_{2}\) is the ratio of their marginal likelihoods, \(B_{12} = p(D \mid M_{1}) / p(D \mid M_{2})\), where \(p(D \mid M) = \int p(D \mid \theta, M)\,\pi(\theta \mid M)\,d\theta\). (II) It measures how strongly the data support one model over the other, and it turns prior odds into posterior odds.(III) concerns the likelihood conditional on the parameters, \(p(D \mid \theta, M)\). That quantity is integrated over the prior inside each marginal likelihood, so the Bayes factor is built from the averaged form and not from the conditional one. The wording of (III) is loose: the conditional likelihood does enter the integral, so a reader could call it involved. We follow the key's reading, which makes options A, C and D false.

Suppose \(X_{1}\) ...... \(X_{n}\) are independent observations from a Poisson distribution with probability mass function :

\[ \begin{aligned} f(x/\lambda) &= \frac{e^{-\lambda}\lambda^{x}}{x!}, \\ x &= 0, 1, 2, ......., \end{aligned} \]

\(\lambda \gt 0\) unkonwn.

Suppose the prior distribution of \(\lambda\) is desired as :

\[ g(\lambda) \propto \lambda,\ e^{-2\lambda},\ \lambda \gt 0. \]

Let \(n = 3\) and the data \((X_{1}, X_{2}, X_{3}) = (1, 1, 2)\). Then, suppose we estimate \(\lambda\) under squared error loss \(L(\lambda, a) = (\lambda - a)^{2}\). Then, the Bayes estimator based on the given data is :

A.1.5
B.1.65
C.2.25
D.1.2

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: D

The Bayes estimate is 1.2.

The prior, read as \(g(\lambda) \propto \lambda e^{-2\lambda}\) (the comma in the stem stands for multiplication), is a gamma density with shape 2 and rate 2. The Poisson likelihood of the data is proportional to \(e^{-3\lambda}\lambda^{1 + 1 + 2}\). Multiplying:

\[ \pi(\lambda|x) \propto \lambda^{5}e^{-5\lambda} \]

a gamma posterior with shape 6 and rate 5. Under squared error loss the Bayes estimator is the posterior mean, \(6/5 = 1.2\).

None of the other values is the posterior mean. Ignoring the prior would give the sample mean 4/3, and the posterior mode, \((6 - 1)/5 = 1\), is not offered either; the prior pulls the estimate below the sample mean because its own mean is 1.

Discuss in Forum

Let \(X_{1}, X_{2}, \ldots, X_{n}\) be i.i.d. Bernoulli \((\theta)\). Let the loss function for the problem of estimation of \(\theta\) be given by

\[ L(\theta, a) = \frac{(\theta - a)^{2}}{\theta(1 - \theta)}. \]

Let the pdf of the prior for \(\theta\) (\(\pi(\theta)\)) be given by \(\pi(\theta) = 1\), \(0 \le \theta \le 1\).

Let \(\overline{X}\) be the sample mean. Then :

A.\(\overline{X}\) is Bayes but not minimax
B.\(\overline{X}\) is not Bayes but it is minimax
C.\(\overline{X}\) is both Bayes and minimax
D.\(\overline{X}\) is not admissible

Exam Year(s)

MH SET - 2021, SET Exam - 2021,

Answer: C

\(\bar{X}\) is both Bayes and minimax.

Under the weighted loss \(L = (\theta - a)^{2}/[\theta(1 - \theta)]\), the Bayes estimator is \(E[\theta w(\theta) \mid x]/E[w(\theta) \mid x]\) with \(w = 1/[\theta(1 - \theta)]\). With the uniform prior the posterior is Beta(s + 1, n - s + 1), where \(s = \sum x_{i}\); multiplying by \(w\) gives a Beta(s, n - s) kernel, whose mean is \(s/n = \bar{X}\). So \(\bar{X}\) is Bayes.

Its risk is constant:

\[ \begin{aligned} \frac{E(\bar{X} - \theta)^{2}}{\theta(1 - \theta)} &= \frac{\theta(1 - \theta)/n}{\theta(1 - \theta)} \\ &= \frac{1}{n} \end{aligned} \]

A Bayes estimator with constant risk is minimax, and a unique Bayes estimator is admissible, so A, B and D are wrong.

Discuss in Forum

Suppose given \(\theta\), the random variable X follows a normal distribution with mean \(\theta\) and variance 1. Let the prior density of \(\theta\) be Bernoulli with parameter 1/2. Then the posterior distribution of \(\theta\) is :

A.\(2e^{-1/2\,(x - \theta)^{2}}\)
B.\(\frac{1}{2}e^{-1/2\,(x - \theta)^{2}}\)
C.\(\dfrac{e^{x\theta - \theta^{2}/2}}{1 + e^{x - 1/2}}\)
D.\(\dfrac{e^{x^{2} + x\theta - \theta^{2}/2}}{1 + e^{x - 1/2}}\)

Exam Year(s)

MH SET - 2023, SET Exam - 2023,

Answer: C

The posterior is \(\dfrac{e^{x\theta - \theta^{2}/2}}{1 + e^{x - 1/2}}\), for \(\theta \in \{0, 1\}\).

The prior puts mass 1/2 on each of \(\theta = 0\) and \(\theta = 1\), so the posterior is proportional to the likelihood:

\[ \begin{aligned} \pi(\theta \mid x) &\propto e^{-(x - \theta)^{2}/2} \propto e^{x\theta - \theta^{2}/2} \\ \text{at } \theta = 0 &: 1,\quad \text{at } \theta = 1 : e^{x - 1/2} \end{aligned} \]

after dropping the factor \(e^{-x^{2}/2}\), which does not depend on \(\theta\). Normalising by the sum \(1 + e^{x - 1/2}\) gives the third option.

The first two options are unnormalised likelihoods that do not sum to 1 over \(\theta\). The last keeps an \(e^{x^{2}}\) factor in the numerator that should have cancelled, so its two values do not add to 1.

Discuss in Forum

Suppose we have 10 observations \(\underset{\sim}{y} = (y_{1}, y_{2}, \ldots, y_{10})\) from Poisson \((\lambda)\) and we assume a Gamma \((a, b)\) prior distribution for \(\lambda\).

\[ \begin{aligned} \Big(p(\lambda) &= \{\Gamma(a)\}^{-1} b^{a} e^{-b\lambda} \lambda^{a-1}, \\ &\quad a \gt 0,\ b \gt 0\Big) \end{aligned} \]

Then the posterior predictive distribution \(p(y_{\text{new}} / \underset{\sim}{y})\) of a new observation \(y_{\text{new}}\) is given by :

A.Gamma \(\left(a + \sum_{i=1}^{10} y_{i},\ b + 10\right)\)
B.Negative Binomial \(\left(a + \sum_{i=1}^{10} y_{i},\ b + 10\right)\)
C.Gamma \(\left(b + 10,\ a + \sum_{i=1}^{10} y_{i}\right)\)
D.Poisson \(\left(a + \sum_{i=1}^{10} y_{i}\right)\)

Exam Year(s)

MH SET - 2023, SET Exam - 2023,

Answer: B

The posterior predictive distribution is Negative Binomial \(\left(a + \sum y_{i},\ b + 10\right)\).

With a Gamma \((a, b)\) prior (rate \(b\)) and 10 Poisson observations, the posterior is Gamma \((a + \sum y_{i},\ b + 10)\), by conjugacy. A new observation is Poisson \((\lambda)\) given \(\lambda\), and mixing a Poisson over a gamma distribution gives a negative binomial:

\[ \begin{aligned} &p(y_{\text{new}} \mid \underset{\sim}{y}) \\ &= \binom{\alpha + y - 1}{y} \left(\frac{\beta}{\beta + 1}\right)^{\alpha} \\ &\quad \times \left(\frac{1}{\beta + 1}\right)^{y} \end{aligned} \]

with \(\alpha = a + \sum y_{i}\) and \(\beta = b + 10\), the parameters named in the option.

The two gamma options describe the posterior of \(\lambda\) (one with its parameters swapped), not the distribution of a new count, which must be discrete. The Poisson option ignores the uncertainty in \(\lambda\), which is exactly what makes the predictive distribution overdispersed.

Discuss in Forum

Suppose

\[ P(Y=y/\theta)=\binom{n}{y}\theta^{y}(1-\theta)^{n-y}, \]

\(y \in \{0, 1, 2, \ldots, n\}\). If \(\theta\) ~ uniform (0, 1), then the posterior distribution will be :

A.Uniform (0, y)
B.Beta (y, n − y)
C.Beta (y + 1, n − y + 1)
D.Uniform (0, 1)

Exam Year(s)

MH SET - 2024, SET Exam - 2024,

Answer: C

The posterior is Beta(y + 1, n − y + 1).

The uniform prior on (0, 1) has density 1, so the posterior is proportional to the likelihood:

\[ \pi(\theta \mid y) \propto \theta^{y}(1 - \theta)^{n-y} \]

This is the kernel of a Beta distribution with parameters \(y + 1\) and \(n - y + 1\) (the Beta(a, b) kernel is \(\theta^{a-1}(1 - \theta)^{b-1}\)).

Beta(y, n − y) drops the 1 contributed by the uniform prior, which is Beta(1, 1). The uniform distributions ignore the data.

Discuss in Forum

Let X ~ Poisson (\(\lambda\)) and a single realization of X resulted in 3. If \(\lambda\) has a prior distribution

\[ P(\lambda) = q\, \lambda\, e^{-3\lambda},\ \lambda \gt 0, \]

then the posterior mean is given by :

A.4/5
B.3/5
C.5/4
D.2/5

Exam Year(s)

MH SET - 2024, SET Exam - 2024,

Answer: C

The posterior mean is 5/4.

The prior \(q\lambda e^{-3\lambda}\) is a Gamma density with shape 2 and rate 3. With one Poisson observation x = 3 the likelihood is proportional to \(\lambda^{3}e^{-\lambda}\), so

\[ \begin{aligned} \pi(\lambda \mid x) &\propto \lambda^{3}e^{-\lambda} \cdot \lambda e^{-3\lambda} \\ &= \lambda^{4}e^{-4\lambda} \end{aligned} \]

which is Gamma with shape 5 and rate 4. Its mean is \(5/4\).

4/5 inverts the shape and rate, and 3/5 and 2/5 use the wrong shape or rate.

Discuss in Forum

Which of the following statements about the Bayes factor is true ?

(I) Bayes factor involves marginal likelihood.
(II) Bayes factor expresses the support given by the data for one or other of the models.
(III) Bayes factor involves the conditional likelihood

A.Only (I) and (III)
B.Only (I) and (II)
C.Only (II) and (III)
D.All are true

Exam Year(s)

MH SET - 2025, SET Exam - 2025,

Answer: B

The key accepts (I) and (II) as true and (III) as false.

(I) The Bayes factor for models \(M_{1}\) and \(M_{2}\) is the ratio of their marginal likelihoods, \(B_{12} = p(D \mid M_{1}) / p(D \mid M_{2})\), where \(p(D \mid M) = \int p(D \mid \theta, M)\,\pi(\theta \mid M)\,d\theta\). (II) It measures how strongly the data support one model over the other, and it turns prior odds into posterior odds.

(III) concerns the likelihood conditional on the parameters, \(p(D \mid \theta, M)\). That quantity is integrated over the prior inside each marginal likelihood, so the Bayes factor is built from the averaged form and not from the conditional one. The wording of (III) is loose: the conditional likelihood does enter the integral, so a reader could call it involved. We follow the key's reading, which makes options A, C and D false.

Bayesian Inference Questions and Answers

Topics

Other Subjects

Tags: Bayesian Inference Questions and Answers

About Us

Bayesian Inference Questions and Answers

Topics

Other Subjects

Tags: Bayesian Inference Questions and Answers

About Us

Stay informed about all important Dates & Notifications