Paper 1, Section I, G

Coding and Cryptography | Part II, 2019

Let XX and YY be discrete random variables taking finitely many values. Define the conditional entropy H(X∣Y)H(X \mid Y). Suppose ZZ is another discrete random variable taking values in a finite alphabet, and prove that

H(X∣Y)⩽H(X∣Y,Z)+H(Z)H(X \mid Y) \leqslant H(X \mid Y, Z)+H(Z)

[You may use the equality H(X,Y)=H(X∣Y)+H(Y)H(X, Y)=H(X \mid Y)+H(Y) and the inequality H(X∣Y)⩽H(X \mid Y) \leqslant H(X).]H(X) .]

State and prove Fano's inequality.

Typos? Please submit corrections to this page on GitHub.