⧼exchistory⧽
BBy Bot
Jun 09'24
[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

Which of the following matrices are transition matrices for regular Markov chains?

  • [math]\mat {P} = \pmatrix{ .5 & .5 \cr .5 & .5 }[/math]
  • [math]\mat {P} = \pmatrix{ .5 & .5 \cr 1 & 0 }[/math]
  • [math]\mat {P} = \pmatrix{ 1/3 & 0 & 2/3 \cr 0 & 1 & 0 \cr 0 & 1/5 & 4/5}[/math]
  • [math]\mat {P} = \pmatrix{ 0 & 1 \cr 1 & 0}[/math]
  • [math]\mat {P} = \pmatrix{ 1/2 & 1/2 & 0 \cr 0 & 1/2 & 1/2 \cr 1/3 & 1/3 & 1/3}[/math]
BBy Bot
Jun 09'24
[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

Consider the Markov chain with transition matrix

[[math]] \mat {P} = \pmatrix{ 1/2 & 1/3 & 1/6 \cr3/4 & 0 & 1/4 \cr 0 & 1 & 0}\ . [[/math]]

  • Show that this is a regular Markov chain.
  • The process is started in state 1; find the probability that it is in state 3 after two steps.
  • Find the limiting probability vector [math]\mat{w}[/math].
BBy Bot
Jun 09'24
[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

Consider the Markov chain with general [math]2 \times 2[/math]

transition matrix

[[math]] \mat {P} = \pmatrix{ 1 - a & a \cr b & 1 - b}\ . [[/math]]

  • Under what conditions is [math]\mat{P}[/math] absorbing?
  • Under what conditions is [math]\mat{P}[/math] ergodic but not regular?
  • Under what conditions is [math]\mat{P}[/math] regular?
BBy Bot
Jun 09'24
[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

Find the fixed probability vector [math]\mat{w}[/math] for the matrices in Exercise Exercise that are ergodic.

BBy Bot
Jun 09'24
[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

Find the fixed probability vector [math]\mat{w}[/math] for each of the following regular matrices.

  • [math]\mat {P} = \pmatrix{ .75 & .25 \cr .5 & .5}[/math]. \smallskip
  • [math]\mat {P} = \pmatrix{ .9 & .1 \cr .1 & .9}[/math]. \smallskip
  • [math]\mat {P} = \pmatrix{ 3/4 & 1/4 & 0 \cr 0 & 2/3 & 1/3 \cr 1/4 & 1/4 & 1/2}[/math]. \smallskip
BBy Bot
Jun 09'24
[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

Consider the Markov chain with transition matrix in Exercise, with [math]a = b = 1[/math]. Show that this chain is ergodic but not regular.

Find the fixed probability vector and interpret it. Show that [math]\mat {P}^n[/math] does not tend to a limit, but that

[[math]] \mat {A}_n = \frac{\mat {I} + \mat {P} + \mat {P}^2 +\cdots + \mat {P}^n}{n + 1} [[/math]]

does.

BBy Bot
Jun 09'24
[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

Consider the Markov chain with transition matrix of Exercise, with [math]a = 0[/math] and [math]b = 1/2[/math]. Compute directly the unique fixed probability vector, and use your result to prove that the chain is not ergodic.

BBy Bot
Jun 09'24
[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

Show that the matrix

[[math]] \mat {P} = \pmatrix{ 1 & 0 & 0 \cr 1/4 & 1/2 & 1/4 \cr 0 & 0 & 1} [[/math]]

has more than one fixed probability vector. Find the matrix that [math]\mat {P}^n[/math] approaches as [math]n \to \infty[/math], and verify that it is not a matrix all of whose rows are the same.

BBy Bot
Jun 09'24
[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

Prove that, if a 3-by-3 transition matrix has the property that its column sums are 1, then [math](1/3, 1/3, 1/3)[/math] is a fixed probability vector. State a similar result for [math]n[/math]-by-[math]n[/math] transition matrices. Interpret these results for ergodic chains.

BBy Bot
Jun 09'24
[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

Is the Markov chain in Example ergodic?