(a) State the strong law of large numbers. State the central limit theorem.

(b) Assuming whatever regularity conditions you require, show that if $\hat{\theta}_{n} \equiv \hat{\theta}_{n}\left(X_{1}, \ldots, X_{n}\right)$ is the maximum-likelihood estimator of the unknown parameter $\theta$ based on an independent identically distributed sample of size $n$, then under $P_{\theta}$

$$\sqrt{n}\left(\hat{\theta}_{n}-\theta\right) \rightarrow N\left(0, J(\theta)^{-1}\right) \quad \text { in distribution }$$

as $n \rightarrow \infty$, where $J(\theta)$ is a matrix which you should identify. A rigorous derivation is not required.

(c) Suppose that $X_{1}, X_{2}, \ldots$ are independent binomial $\operatorname{Bin}(1, \theta)$ random variables. It is required to test $H_{0}: \theta=\frac{1}{2}$ against the alternative $H_{1}: \theta \in(0,1)$. Show that the construction of a likelihood-ratio test leads us to the statistic

$$T_{n}=2 n\left\{\hat{\theta}_{n} \log \hat{\theta}_{n}+\left(1-\hat{\theta}_{n}\right) \log \left(1-\hat{\theta}_{n}\right)+\log 2\right\}$$

where $\hat{\theta}_{n} \equiv n^{-1} \sum_{k=1}^{n} X_{k}$. Stating clearly any result to which you appeal, for large $n$, what approximately is the distribution of $T_{n}$ under $H_{0}$ ? Writing $\hat{\theta}_{n}=\frac{1}{2}+Z_{n}$, and assuming that $Z_{n}$ is small, show that

$$T_{n} \simeq 4 n Z_{n}^{2}$$

Using this and the central limit theorem, briefly justify the approximate distribution of $T_{n}$ given by asymptotic maximum-likelihood theory. What could you say if the assumption that $Z_{n}$ is small failed?