[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

Consider the Markov chain with transition matrix in

Exercise \ref{exer 11.3.3}, with [math]a = b = 1[/math]. Show that this chain is ergodic but not regular. Find the fixed probability vector and interpret it. Show that [math]\mat {P}^n[/math] does not tend to a limit, but that

[[math]] \mat {A}_n = \frac{\mat {I} + \mat {P} + \mat {P}^2 +\cdots + \mat {P}^n}{n + 1} [[/math]]

does.