[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

A rat runs through the maze shown in Figure. At each step it leaves the room it is in by choosing at random one of the doors out of the room.

• Give the transition matrix [math]\mat{P}[/math] for this Markov chain.
• Show that it is an ergodic chain but not a regular chain.
• Find the fixed vector.
• Find the expected number of steps before reaching Room 5 for the first time, starting in Room 1.