4.II.29I

Optimization and Control | Part II, 2007

Consider the scalar controllable linear system, whose state XnX_{n} evolves by

Xn+1=Xn+Un+εn+1X_{n+1}=X_{n}+U_{n}+\varepsilon_{n+1}

with observations YnY_{n} given by

Yn+1=Xn+ηn+1Y_{n+1}=X_{n}+\eta_{n+1}

Here, UnU_{n} is the control variable, which is to be determined on the basis of the observations up to time nn, and εn,ηn\varepsilon_{n}, \eta_{n} are independent N(0,1)N(0,1) random variables. You wish to minimize the long-run average expected cost, where the instantaneous cost at time nn is Xn2+Un2X_{n}^{2}+U_{n}^{2}. You may assume that the optimal control in equilibrium has the form Un=KX^nU_{n}=-K \hat{X}_{n}, where X^n\hat{X}_{n} is given by a recursion of the form

X^n+1=X^n+Un+H(Yn+1X^n)\hat{X}_{n+1}=\hat{X}_{n}+U_{n}+H\left(Y_{n+1}-\hat{X}_{n}\right)

and where HH is chosen so that Δn=XnX^n\Delta_{n}=X_{n}-\hat{X}_{n} is independent of the observations up to time nn. Show that K=H=(51)/2=2/(5+1)K=H=(\sqrt{5}-1) / 2=2 /(\sqrt{5}+1), and determine the minimal long-run average expected cost. You are not expected to simplify the arithmetic form of your answer but should show clearly how you have obtained it.

Typos? Please submit corrections to this page on GitHub.