Suppose that the random vector $\mathbf{X}=\left(X_{1}, \ldots, X_{n}\right)$ has a distribution over $\mathbb{R}^{n}$ depending on a real parameter $\theta$, with everywhere positive density function $p(\mathbf{x} \mid \theta)$. Define the maximum likelihood estimator $\hat{\theta}$, the score variable $U$, the observed information $\hat{j}$ and the expected (Fisher) information $I$ for the problem of estimating $\theta$ from $\mathbf{X}$.

For the case where the $\left(X_{i}\right)$ are independent and identically distributed, show that, as $n \rightarrow \infty, I^{-1 / 2} U \stackrel{d}{\rightarrow} \mathcal{N}(0,1)$. [You may assume sufficient conditions to allow interchange of integration over the sample space and differentiation with respect to the parameter.] State the asymptotic distribution of $\hat{\theta}$.

The random vector $\mathbf{X}=\left(X_{1}, \ldots, X_{n}\right)$ is generated according to the rule

$$X_{i+1}=\theta X_{i}+E_{i}$$

where $X_{0}=1$ and the $\left(E_{i}\right)$ are independent and identically distributed from the standard normal distribution $\mathcal{N}(0,1)$. Write down the likelihood function for $\theta$ based on data $\mathbf{x}=\left(x_{1}, \ldots, x_{n}\right)$, find $\hat{\theta}$ and $\hat{j}$ and show that the pair $(\hat{\theta}, \hat{j})$ forms a minimal sufficient statistic.

A Bayesian uses the improper prior density $\pi(\theta) \propto 1$. Show that, in the posterior, $S(\theta-\hat{\theta})$ (where $S$ is a statistic that you should identify) has the same distribution as $E_{1}$.