Let $\mathcal{A}$ and $\mathcal{B}$ be alphabets of sizes $m$ and a respectively. What does it mean to say that $c: \mathcal{A} \rightarrow \mathcal{B}^{*}$ is a decodable code? State Kraft's inequality.

Suppose that a source emits letters from the alphabet $\mathcal{A}=\{1,2, \ldots, m\}$, each letter $j$ occurring with (known) probability $p_{j}>0$. Let $S$ be the codeword-length random variable for a decodable code $c: \mathcal{A} \rightarrow \mathcal{B}^{*}$, where $|\mathcal{B}|=a$. It is desired to find a decodable code that minimizes the expected value of $a^{S}$. Establish the lower bound $\mathbb{E}\left(a^{S}\right) \geqslant\left(\sum_{j=1}^{m} \sqrt{p_{j}}\right)^{2}$, and characterise when equality occurs. [Hint. You may use without proof the Cauchy-Schwarz inequality, that (for positive $x_{i}, y_{i}$ )

$$\sum_{i=1}^{m} x_{i} y_{i} \leqslant\left(\sum_{i=1}^{m} x_{i}^{2}\right)^{1 / 2}\left(\sum_{i=1}^{m} y_{i}^{2}\right)^{1 / 2}$$

with equality if and only if $x_{i}=\lambda y_{i}$ for all i.]