exercise:60b4d769a6: Difference between revisions

From Stochiki
(Created page with "<div class="d-none"><math> \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}</math></div> Consider a game played as follows: You are given a regular Markov chain with transition matrix <math>\mat P</math>, fixed probability vector <math>\mat{w}</math>, and a payoff function <math>\mat f</math> which assigns to each state <math>s_i</mat...")
 
No edit summary
Line 5: Line 5:
\newcommand{\secstoprocess}{\all}
\newcommand{\secstoprocess}{\all}
\newcommand{\NA}{{\rm NA}}
\newcommand{\NA}{{\rm NA}}
\newcommand{\mathds}{\mathbb}</math></div> Consider a game played as follows: You are given a
\newcommand{\mathds}{\mathbb}</math></div> Consider a game played as follows: You are given a regular Markov chain with transition matrix
regular Markov chain
<math>\mat P</math>, fixed probability vector <math>\mat{w}</math>, and a payoff function <math>\mat f</math> which assigns to each
with transition matrix
<math>\mat P</math>, fixed probability vector <math>\mat{w}</math>, and a payoff function <math>\mat f</math>
which assigns to each
state <math>s_i</math> an amount <math>f_i</math> which may be positive or negative.  Assume that
state <math>s_i</math> an amount <math>f_i</math> which may be positive or negative.  Assume that
<math>\mat {w}\mat {f} =
<math>\mat {w}\mat {f} =

Revision as of 01:29, 15 June 2024

[math] \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}[/math]

Consider a game played as follows: You are given a regular Markov chain with transition matrix

[math]\mat P[/math], fixed probability vector [math]\mat{w}[/math], and a payoff function [math]\mat f[/math] which assigns to each state [math]s_i[/math] an amount [math]f_i[/math] which may be positive or negative. Assume that [math]\mat {w}\mat {f} = 0[/math]. You watch this Markov chain as it evolves, and every time you are in state [math]s_i[/math] you receive an amount [math]f_i[/math]. Show that your expected winning after [math]n[/math] steps can be represented by a column vector [math]\mat{g}^{(n)}[/math], with

[[math]] \mat{g}^{(n)} = (\mat {I} + \mat {P} + \mat {P}^2 +\cdots+ \mat {P}^n) \mat {f}. [[/math]]

Show that as [math]n \to \infty[/math], [math]\mat {g}^{(n)} \to \mat {g}[/math] with [math]\mat {g} = \mat {Z} \mat {f}[/math].