Discussion
Let X and Y be random variables with joint distribution :
\[ \Pr(X = i, Y = j) = \begin{cases} 1/6 & \text{for } i = 5,6 \text{ and } j = 3,6,9 \\ 0 & \text{otherwise} \end{cases} \]
Then the mutual information I (X, Y) = .................. .
Answer: A
The mutual information is 0.
The joint distribution puts probability 1/6 on each of the 2 × 3 = 6 pairs. The marginals are therefore uniform:
\[ \begin{aligned} P(X = i) &= 3 \times \tfrac{1}{6} = \tfrac{1}{2} \\ P(Y = j) &= 2 \times \tfrac{1}{6} = \tfrac{1}{3} \end{aligned} \]
Since \(\tfrac{1}{2} \times \tfrac{1}{3} = \tfrac{1}{6}\) for every pair, \(P(X, Y) = P(X)P(Y)\): X and Y are independent, and independent variables share no information.
\[ \begin{aligned} I(X, Y) &= H(X) + H(Y) - H(X, Y) \\ &= \log 2 + \log 3 - \log 6 = 0 \end{aligned} \]
The distractors are the separate entropies: log(2) is H(X), log(3) is H(Y), and log(6) is the joint entropy H(X, Y).
Post your comment
Please log in or register to post a comment.
Examination Categories
- SSC Exams
- Banking Exams
- Teaching Exams
- Civil Services Exams
- Railways Exams
- Engineering Recruitment Exams
- Defence Exams
- State Government Exams
- Police Exams
- Insurance Exams
- Nursing Exams
- Other Government Exams
- NRA CET
- PG Entrance Exams
- Campus Placement
- MBA Entrance Exams
- CUET and UG Entrance
- Judiciary Exams
- Regulatory Body Exams
- Software Development