We consider a system of two queues in tandem, as follows. Customers arrive in the first queue at rate $\lambda$. Each arriving customer is immediately served by one of infinitely many servers at rate $\mu_{1}$. Immediately after service, customers join a single-server second queue which operates on a first-come, first-served basis, and has a service rate $\mu_{2}$. After service in this second queue, each customer returns to the first queue with probability $0<1-p<1$, and otherwise leaves the system forever. A schematic representation is given below:

(a) Let $M_{t}$ and $N_{t}$ denote the number of customers at time $t$ in queues number 1 and 2 respectively, including those currently in service at time $t$. Give the transition rates of the Markov chain $\left(M_{t}, N_{t}\right)_{t \geqslant 0}$.

(b) Write down an equation satisfied by any invariant measure $\pi$ for this Markov chain. Let $\alpha>0$ and $\beta \in(0,1)$. Define a measure $\pi$ by

$$\pi(m, n):=e^{-\alpha} \frac{\alpha^{m}}{m !} \beta^{n}(1-\beta), \quad m, n \in\{0,1, \ldots\}$$

Show that it is possible to find $\alpha>0, \beta \in(0,1)$ so that $\pi$ is an invariant measure of $\left(M_{t}, N_{t}\right)_{t \geqslant 0}$, if and only if $\lambda<\mu_{2} p$. Give the values of $\alpha$ and $\beta$ in this case.

(c) Assume now that $\lambda p>\mu_{2}$. Show that the number of customers is not positive recurrent.

[Hint. One way to solve the problem is as follows. Assume it is positive recurrent. Observe that $M_{t}$ is greater than a $M / M / \infty$ queue with arrival rate $\lambda$. Deduce that $N_{t}$ is greater than a $M / M / 1$ queue with arrival rate $\lambda p$ and service rate $\mu_{2}$. You may use without proof the fact that the departure process from the first queue is then, at equilibrium, a Poisson process with rate $\lambda$, and you may use without proof properties of thinned Poisson processes.]