Suppose that $B$ is a non-empty subset of the statespace $I$ of a Markov chain $X$ with transition matrix $P$, and let $\tau \equiv \inf \left\{n \geqslant 0: X_{n} \in B\right\}$, with the convention that inf $\emptyset=\infty$. If $h_{i}=P\left(\tau<\infty \mid X_{0}=i\right)$, show that the equations

(a)

$$g_{i} \geqslant(P g)_{i} \equiv \sum_{j \in I} p_{i j} g_{j} \geqslant 0 \quad \forall i$$

$$g_{i}=1 \quad \forall i \in B$$

are satisfied by $g=h$.

If $g$ satisfies (a), prove that $g$ also satisfies

$(c)$

$$g_{i} \geqslant(\tilde{P} g)_{i} \quad \forall i,$$

where

$$\tilde{p}_{i j}=\left\{\begin{array}{cl} p_{i j} & (i \notin B), \\ \delta_{i j} & (i \in B) \end{array}\right.$$

By interpreting the transition matrix $\tilde{P}$, prove that $h$ is the minimal solution to the equations (a), (b).

Now suppose that $P$ is irreducible. Prove that $P$ is recurrent if and only if the only solutions to (a) are constant functions.