exercise:60b4d769a6: Difference between revisions
(Created page with "<div class="d-none"><math> \newcommand{\NA}{{\rm NA}} \newcommand{\mat}[1]{{\bf#1}} \newcommand{\exref}[1]{\ref{##1}} \newcommand{\secstoprocess}{\all} \newcommand{\NA}{{\rm NA}} \newcommand{\mathds}{\mathbb}</math></div> Consider a game played as follows: You are given a regular Markov chain with transition matrix <math>\mat P</math>, fixed probability vector <math>\mat{w}</math>, and a payoff function <math>\mat f</math> which assigns to each state <math>s_i</mat...") |
No edit summary |
||
Line 5: | Line 5: | ||
\newcommand{\secstoprocess}{\all} | \newcommand{\secstoprocess}{\all} | ||
\newcommand{\NA}{{\rm NA}} | \newcommand{\NA}{{\rm NA}} | ||
\newcommand{\mathds}{\mathbb}</math></div> Consider a game played as follows: You are given a | \newcommand{\mathds}{\mathbb}</math></div> Consider a game played as follows: You are given a regular Markov chain with transition matrix | ||
regular Markov chain | <math>\mat P</math>, fixed probability vector <math>\mat{w}</math>, and a payoff function <math>\mat f</math> which assigns to each | ||
with transition matrix | |||
<math>\mat P</math>, fixed probability vector <math>\mat{w}</math>, and a payoff function <math>\mat f</math> | |||
which assigns to each | |||
state <math>s_i</math> an amount <math>f_i</math> which may be positive or negative. Assume that | state <math>s_i</math> an amount <math>f_i</math> which may be positive or negative. Assume that | ||
<math>\mat {w}\mat {f} = | <math>\mat {w}\mat {f} = |
Revision as of 01:29, 15 June 2024
Consider a game played as follows: You are given a regular Markov chain with transition matrix
[math]\mat P[/math], fixed probability vector [math]\mat{w}[/math], and a payoff function [math]\mat f[/math] which assigns to each state [math]s_i[/math] an amount [math]f_i[/math] which may be positive or negative. Assume that [math]\mat {w}\mat {f} = 0[/math]. You watch this Markov chain as it evolves, and every time you are in state [math]s_i[/math] you receive an amount [math]f_i[/math]. Show that your expected winning after [math]n[/math] steps can be represented by a column vector [math]\mat{g}^{(n)}[/math], with
Show that as [math]n \to \infty[/math], [math]\mat {g}^{(n)} \to \mat {g}[/math] with [math]\mat {g} = \mat {Z} \mat {f}[/math].