[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

(Crowell) Let [math]\mat{P}[/math]

be the transition matrix of an ergodic Markov chain. Show that

[[math]] (\mat {I} + \mat {P} +\cdots+ \mat {P}^{n - 1})(\mat {I} - \mat {P} + \mat {W}) = \mat {I} - \mat {P}^n + n\mat {W}\ , [[/math]]

and from this show that

[[math]] \frac{\mat {I} + \mat {P} +\cdots+ \mat {P}^{n - 1}}n \to \mat {W}\ , [[/math]]

as [math]n \rightarrow \infty[/math].

Notes