[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

List the states of the chain. Hint: There are six. Compute the transition probabilities. Find the fundamental matrix [math]\mat{N}[/math], [math]\mat{N}\mat{c}[/math], and [math]\mat{N}\mat{R}[/math].