Paper 4, Section I, H

Markov Chains | Part IB, 2019

For a Markov chain XX on a state space SS with u,v∈Su, v \in S, we let puv(n)p_{u v}(n) for n∈{0,1,…}n \in\{0,1, \ldots\} be the probability that Xn=vX_{n}=v when X0=uX_{0}=u.

(a) Let XX be a Markov chain. Prove that if XX is recurrent at a state vv, then ∑n=0∞pvv(n)=∞\sum_{n=0}^{\infty} p_{v v}(n)=\infty. [You may use without proof that the number of returns of a Markov chain to a state vv when starting from vv has the geometric distribution.]

(b) Let XX and YY be independent simple symmetric random walks on Z2\mathbb{Z}^{2} starting from the origin 0 . Let Z=∑n=0∞1{Xn=Yn}Z=\sum_{n=0}^{\infty} \mathbf{1}_{\left\{X_{n}=Y_{n}\right\}}. Prove that E[Z]=∑n=0∞p00(2n)\mathbb{E}[Z]=\sum_{n=0}^{\infty} p_{00}(2 n) and deduce that E[Z]=∞\mathbb{E}[Z]=\infty. [You may use without proof that pxy(n)=pyx(n)p_{x y}(n)=p_{y x}(n) for all x,y∈Z2x, y \in \mathbb{Z}^{2} and n∈Nn \in \mathbb{N}, and that XX is recurrent at 0.]

Typos? Please submit corrections to this page on GitHub.