Two scalar systems have dynamics

$$x_{t+1}=x_{t}+u_{t}+\epsilon_{t}, \quad y_{t+1}=y_{t}+w_{t}+\eta_{t},$$

where $\left\{\epsilon_{t}\right\}$ and $\left\{\eta_{t}\right\}$ are independent sequences of independent and identically distributed random variables of mean 0 and variance 1 . Let

$$F(x)=\inf _{\pi} \mathbb{E}\left[\sum_{t=0}^{\infty}\left(x_{t}^{2}+u_{t}^{2}\right)(2 / 3)^{t} \mid x_{0}=x\right]$$

where $\pi$ is a policy in which $u_{t}$ depends on only $x_{0}, \ldots, x_{t}$.

Show that $G(x)=P x^{2}+d$ is a solution to the optimality equation satisfied by $F(x)$, for some $P$ and $d$ which you should find.

Find the optimal controls.

State a theorem that justifies $F(x)=G(x)$.

For each of the two cases (a) $\lambda=0$ and (b) $\lambda=1$, find controls $\left\{u_{t}, w_{t}\right\}$ which minimize

$$\mathbb{E}\left[\sum_{t=0}^{\infty}\left(x_{t}^{2}+2 \lambda x_{t} y_{t}+y_{t}^{2}+u_{t}^{2}+w_{t}^{2}\right)(2 / 3+\lambda / 12)^{t} \mid x_{0}=x, y_{0}=y\right]$$