Discussion
Let X and Y be random variables with joint distribution :
\[ \Pr(X = i, Y = j) = \begin{cases} 1/6 & \text{for } i = 5,6 \text{ and } j = 3,6,9 \\ 0 & \text{otherwise} \end{cases} \]
Then the conditional entropy H(X|Y) = .................. .
Answer: B
The conditional entropy is log(2).
The joint distribution spreads probability 1/6 evenly over the six pairs with \(i \in \{5, 6\}\) and \(j \in \{3, 6, 9\}\). So X is uniform on two values and Y uniform on three, and every joint probability is the product of the marginals:
\[ \begin{aligned} P(X=i) &= \tfrac{1}{2}, \quad P(Y=j) = \tfrac{1}{3} \\ P(X=i, Y=j) &= \tfrac{1}{2} \cdot \tfrac{1}{3} = \tfrac{1}{6} \end{aligned} \]
X and Y are therefore independent, and knowing Y tells us nothing about X:
\[ H(X \mid Y) = H(X) = \log 2 \]
0 would mean Y determines X completely. log(3) is H(Y), and log(6) is the joint entropy H(X, Y) = log 2 + log 3.
Post your comment
Please log in or register to post a comment.
Examination Categories
- SSC Exams
- Banking Exams
- Teaching Exams
- Civil Services Exams
- Railways Exams
- Engineering Recruitment Exams
- Defence Exams
- State Government Exams
- Police Exams
- Insurance Exams
- Nursing Exams
- Other Government Exams
- NRA CET
- PG Entrance Exams
- Campus Placement
- MBA Entrance Exams
- CUET and UG Entrance
- Judiciary Exams
- Regulatory Body Exams
- Software Development