2.II.20D
Consider a Markov chain with state space and transition probabilities given by
with , otherwise, where and .
For each , let
that is, the probability that the chain ever hits the state 0 given that it starts in state . Write down the equations satisfied by the probabilities and hence, or otherwise, show that they satisfy a second-order recurrence relation with constant coefficients. Calculate for each .
Determine for each value of , whether the chain is transient, null recurrent or positive recurrent and in the last case calculate the stationary distribution.
[Hint: When the chain is positive recurrent, the stationary distribution is geometric.]
Typos? Please submit corrections to this page on GitHub.