[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

Consider the Markov chain with transition matrix

[[math]] \mat {P} = \pmatrix{ 1/2 & 1/3 & 1/6 \cr3/4 & 0 & 1/4 \cr 0 & 1 & 0}\ . [[/math]]

• Show that this is a regular Markov chain.
• The process is started in state 1; find the probability that it is in state 3 after two steps.
• Find the limiting probability vector [math]\mat{w}[/math].