(a) Define the entropy $h(X)$ and the mutual entropy $i(X, Y)$ of random variables $X$ and $Y$. Prove the inequality

$$0 \leqslant i(X, Y) \leqslant \min \{h(X), h(Y)\}$$

[You may assume the Gibbs inequality.]

(b) Let $X$ be a random variable and let $\mathbf{Y}=\left(Y_{1}, \ldots, Y_{n}\right)$ be a random vector.

(i) Prove or disprove by producing a counterexample the inequality

$$i(X, \mathbf{Y}) \leqslant \sum_{j=1}^{n} i\left(X, Y_{j}\right)$$

first under the assumption that $Y_{1}, \ldots, Y_{n}$ are independent random variables, and then under the assumption that $Y_{1}, \ldots, Y_{n}$ are conditionally independent given $X$.

(ii) Prove or disprove by producing a counterexample the inequality

$$i(X, \mathbf{Y}) \geqslant \sum_{j=1}^{n} i\left(X, Y_{j}\right)$$

first under the assumption that $Y_{1}, \ldots, Y_{n}$ are independent random variables, and then under the assumption that $Y_{1}, \ldots, Y_{n}$ are conditionally independent given $X$.