exercise:2ab75d75ba: Difference between revisions
From Stochiki
(Created page with "<div class="d-none"><math> \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}</math></div> Assume that an ergodic Markov chain has states <math>s_1, s_2, \ldots, s_k</math>. Let <math>S^{(n)}_j</math> denote the number of times that the chain is in state <math>s_j</math> in the first <math>n</math> steps. Let <math>\mat{w}</math> deno...") |
No edit summary |
||
| Line 5: | Line 5: | ||
\newcommand{\secstoprocess}{\all} | \newcommand{\secstoprocess}{\all} | ||
\newcommand{\NA}{{\rm NA}} | \newcommand{\NA}{{\rm NA}} | ||
\newcommand{\mathds}{\mathbb}</math></div> Assume that an ergodic Markov chain has states <math>s_1, | \newcommand{\mathds}{\mathbb}</math></div> Assume that an ergodic Markov chain has states <math>s_1,s_2, \ldots,s_k</math>. Let <math>S^{(n)}_j</math> denote the number of times that the chain is in state <math>s_j</math> in the first <math>n</math> steps. Let <math>\mat{w}</math> denote the fixed probability row vector for this chain. Show that, regardless of the starting state, the expected value of <math>S^{(n)}_j</math>, | ||
s_2, \ldots, | divided by <math>n</math>, tends to <math>w_j</math> as <math>n \rightarrow \infty</math>. '' Hint'': If the chain starts in | ||
s_k</math>. Let <math>S^{(n)}_j</math> denote the number of times that the chain is in state | state <math>s_i</math>, then the expected value of <math>S^{(n)}_j</math> is given by the expression | ||
<math>s_j</math> in the | |||
first <math>n</math> steps. Let <math>\mat{w}</math> denote the fixed probability row vector for | |||
this chain. Show | |||
that, regardless of the starting state, the expected value of <math>S^{(n)}_j</math>, | |||
divided by <math>n</math>, | |||
tends to <math>w_j</math> as <math>n \rightarrow \infty</math>. '' Hint'': If the chain starts in | |||
state | |||
<math>s_i</math>, then the expected value of <math>S^{(n)}_j</math> is given by the expression | |||
<math display="block"> | <math display="block"> | ||
\sum_{h = 0}^n p^{(h)}_{ij}\ . | \sum_{h = 0}^n p^{(h)}_{ij}\ . | ||
</math> | </math> | ||
Latest revision as of 01:33, 15 June 2024
[math]
\newcommand{\NA}{{\rm NA}}
\newcommand{\mat}[1]{{\bf#1}}
\newcommand{\exref}[1]{\ref{##1}}
\newcommand{\secstoprocess}{\all}
\newcommand{\NA}{{\rm NA}}
\newcommand{\mathds}{\mathbb}[/math]
Assume that an ergodic Markov chain has states [math]s_1,s_2, \ldots,s_k[/math]. Let [math]S^{(n)}_j[/math] denote the number of times that the chain is in state [math]s_j[/math] in the first [math]n[/math] steps. Let [math]\mat{w}[/math] denote the fixed probability row vector for this chain. Show that, regardless of the starting state, the expected value of [math]S^{(n)}_j[/math],
divided by [math]n[/math], tends to [math]w_j[/math] as [math]n \rightarrow \infty[/math]. Hint: If the chain starts in state [math]s_i[/math], then the expected value of [math]S^{(n)}_j[/math] is given by the expression
[[math]]
\sum_{h = 0}^n p^{(h)}_{ij}\ .
[[/math]]