(i) State and prove Gibbs' inequality.

(ii) A casino offers me the following game: I choose strictly positive numbers $a_{1}, \ldots, a_{n}$ with $\sum_{j=1}^{n} a_{j}=1$. I give the casino my entire fortune $f$ and roll an $n$-sided die. With probability $p_{j}$ the casino returns $u_{j}^{-1} a_{j} f$ for $j=1,2, \ldots, n$. If I intend to play the game many times (staking my entire fortune each time) explain carefully why I should choose $a_{1}, \ldots, a_{n}$ to maximise $\sum_{j=1}^{n} p_{j} \log \left(u_{j}^{-1} a_{j}\right)$.

[You should assume $n \geqslant 2$ and $u_{j}, p_{j}>0$ for each $j .$ ]

(iii) Determine the appropriate $a_{1}, \ldots, a_{n}$. Let $\sum_{i=1}^{n} u_{i}=U$. Show that, if $U<1$, then, in the long run with high probability, my fortune increases. Show that, if $U>1$, the casino can choose $u_{1}, \ldots, u_{n}$ in such a way that, in the long run with high probability, my fortune decreases. Is it true that, if $U>1$, any choice of $u_{1}, \ldots, u_{n}$ will ensure that, in the long run with high probability, my fortune decreases? Why?