Consider the system in scalar variables, for $t=1,2, \ldots, h$ :

$$\begin{aligned} x_{t} &=x_{t-1}+u_{t-1} \\ y_{t} &=x_{t-1}+\eta_{t} \\ \hat{x}_{0} &=x_{0}+\eta_{0} \end{aligned}$$

where $\hat{x}_{0}$ is given, $y_{t}, u_{t}$ are observed at $t$, but $x_{0}, x_{1}, \ldots$ and $\eta_{0}, \eta_{1}, \ldots$ are unobservable, and $\eta_{0}, \eta_{1}, \ldots$ are independent random variables with mean 0 and variance $v$. Define $\hat{x}_{t-1}$ to be the estimator of $x_{t-1}$ with minimum variance amongst all estimators that are unbiased and linear functions of $W_{t-1}=\left(\hat{x}_{0}, y_{1}, \ldots, y_{t-1}, u_{0}, \ldots, u_{t-2}\right)$. Suppose $\hat{x}_{t-1}=a^{T} W_{t-1}$ and its variance is $V_{t-1}$. After observation at $t$ of $\left(y_{t}, u_{t-1}\right)$, a new unbiased estimator of $x_{t-1}$, linear in $W_{t}$, is expressed

$$x_{t-1}^{*}=(1-H) b^{T} W_{t-1}+H y_{t}$$

Find $b$ and $H$ to minimize the variance of $x_{t-1}^{*}$. Hence find $\hat{x}_{t}$ in terms of $\hat{x}_{t-1}, y_{t}, u_{t-1}$, $V_{t-1}$ and $v$. Calculate $V_{h}$.

Suppose $\eta_{0}, \eta_{1}, \ldots$ are Gaussian and thus $\hat{x}_{t}=E\left[x_{t} \mid W_{t}\right]$. Consider minimizing $E\left[x_{h}^{2}+\sum_{t=0}^{h-1} u_{t}^{2}\right]$, under the constraint that the control $u_{t}$ can only depend on $W_{t}$. Show that the value function of dynamic programming for this problem can be expressed

$$F\left(W_{t}\right)=\Pi_{t} \hat{x}_{t}^{2}+\cdots$$

where $F\left(W_{h}\right)=\hat{x}_{h}^{2}+V_{h}$ and $+\cdots$ is independent of $W_{t}$ and linear in $v$.