Let $P$ be a transition matrix on state space $I$. What does it mean for a distribution $\pi$ to be an invariant distribution? What does it mean for $\pi$ and $P$ to be in detailed balance? Show that if $\pi$ and $P$ are in detailed balance, then $\pi$ is an invariant distribution.

(a) Assuming that an invariant distribution exists, state the relationship between this and

(i) the expected return time to a state $i$;

(ii) the expected time spent in a state $i$ between visits to a state $k$.

(b) Let $\left(X_{n}\right)_{n \geqslant 0}$ be a Markov chain with transition matrix $P=\left(p_{i j}\right)_{i, j \in I}$ where $I=\{0,1,2, \ldots\}$. The transition probabilities are given for $i \geqslant 1$ by

$$p_{i j}= \begin{cases}q^{-(i+2)} & \text { if } j=i+1, \\ q^{-i} & \text { if } j=i-1 \\ 1-q^{-(i+2)}-q^{-i} & \text { if } j=i\end{cases}$$

where $q \geqslant 2$. For $p \in(0,1]$ let $p_{01}=p=1-p_{00}$. Compute the following, justifying your answers:

(i) The expected time spent in states $\{2,4,6, \ldots\}$ between visits to state 1 ;

(ii) The expected time taken to return to state 1 , starting from 1 ;

(iii) The expected time taken to hit state 0 starting from $1 .$