Paper 1, Section I, G

Coding and Cryptography | Part II, 2019

Let XX and YY be discrete random variables taking finitely many values. Define the conditional entropy H(XY)H(X \mid Y). Suppose ZZ is another discrete random variable taking values in a finite alphabet, and prove that

H(XY)H(XY,Z)+H(Z)H(X \mid Y) \leqslant H(X \mid Y, Z)+H(Z)

[You may use the equality H(X,Y)=H(XY)+H(Y)H(X, Y)=H(X \mid Y)+H(Y) and the inequality H(XY)H(X \mid Y) \leqslant H(X).]H(X) .]

State and prove Fano's inequality.

Typos? Please submit corrections to this page on GitHub.