We consider a statistical model $\{f(\cdot, \theta): \theta \in \Theta\}$.

(a) Define the maximum likelihood estimator (MLE) and the Fisher information $I(\theta) .$

(b) Let $\Theta=\mathbb{R}$ and assume there exist a continuous one-to-one function $\mu: \mathbb{R} \rightarrow \mathbb{R}$ and a real-valued function $h$ such that

$$\mathbb{E}_{\theta}[h(X)]=\mu(\theta) \quad \forall \theta \in \mathbb{R}$$

(i) For $X_{1}, \ldots, X_{n}$ i.i.d. from the model for some $\theta_{0} \in \mathbb{R}$, give the limit in almost sure sense of

$$\hat{\mu}_{n}=\frac{1}{n} \sum_{i=1}^{n} h\left(X_{i}\right)$$

Give a consistent estimator $\hat{\theta}_{n}$ of $\theta_{0}$ in terms of $\hat{\mu}_{n}$.

(ii) Assume further that $\mathbb{E}_{\theta_{0}}\left[h(X)^{2}\right]<\infty$ and that $\mu$ is continuously differentiable and strictly monotone. What is the limit in distribution of $\sqrt{n}\left(\hat{\theta}_{n}-\theta_{0}\right)$. Assume too that the statistical model satisfies the usual regularity assumptions. Do you necessarily expect $\operatorname{Var}\left(\hat{\theta}_{n}\right) \geqslant\left(n I\left(\theta_{0}\right)\right)^{-1}$ for all $n$ ? Why?

(iii) Propose an alternative estimator for $\theta_{0}$ with smaller bias than $\hat{\theta}_{n}$ if $B_{n}\left(\theta_{0}\right)=$ $\mathbb{E}_{\theta_{0}}\left[\hat{\theta}_{n}\right]-\theta_{0}=\frac{a}{n}+\frac{b}{n^{2}}+O\left(\frac{1}{n^{3}}\right)$ for some $a, b \in \mathbb{R}$ with $a \neq 0$.

(iv) Further to all the assumptions in iii), assume that the MLE for $\theta_{0}$ is of the form

$$\hat{\theta}_{M L E}=\frac{1}{n} \sum_{i=1}^{n} h\left(X_{i}\right)$$

What is the link between the Fisher information at $\theta_{0}$ and the variance of $h(X)$ ? What does this mean in terms of the precision of the estimator and why?

[You may use results from the course, provided you state them clearly.]