Discrete time Martingales

[math] \newcommand{\R}{\mathbb{R}} \newcommand{\A}{\mathcal{A}} \newcommand{\B}{\mathcal{B}} \newcommand{\N}{\mathbb{N}} \newcommand{\C}{\mathbb{C}} \newcommand{\Rbar}{\overline{\mathbb{R}}} \newcommand{\Bbar}{\overline{\mathcal{B}}} \newcommand{\Q}{\mathbb{Q}} \newcommand{\E}{\mathbb{E}} \newcommand{\p}{\mathbb{P}} \newcommand{\one}{\mathds{1}} \newcommand{\0}{\mathcal{O}} \newcommand{\mat}{\textnormal{Mat}} \newcommand{\sign}{\textnormal{sign}} \newcommand{\CP}{\mathcal{P}} \newcommand{\CT}{\mathcal{T}} \newcommand{\CY}{\mathcal{Y}} \newcommand{\F}{\mathcal{F}} \newcommand{\mathds}{\mathbb}[/math]

Recall that the strong law of large numbers tells us, if [math](X_n)_{n\geq 1}[/math] is iid, [math]\E[\vert X_i\vert] \lt \infty[/math] and [math]\E[X_i]=\mu[/math], then

[[math]] \frac{1}{n}S_n\xrightarrow{n\to\infty\atop a.s.}\mu, [[/math]]

with [math]S_n=\sum_{i=1}^nX_i[/math]. We saw that the 0-1 law of Kolmogorov implied that in this case the limit, if it exists, is constant. It is of course of interest to have a framework in which the sequence of r.v.'s converges a.s. to another r.v. This can be achieved in the framework of martingales. In this chapter, we shall consider a probability space [math](\Omega,\F,\p)[/math] as well as an increasing family [math](\F_n)_{n\geq 0}[/math] of sub [math]\sigma[/math]-Algebras of [math]\F[/math], i.e. [math]\F_n\subset\F_{n+1}\subset \F[/math]. Such a sequence is called a filtration. The space [math](\Omega,\F,(\F_n)_{n\geq 0},\p)[/math] is called a filtered probability space. We shall also consider a sequence [math](X_n)_{n\geq 0}[/math] of r.v.'s. Such a sequence is generally called a stochastic process ([math]n[/math] is thought of as time). If for every [math]n\geq0[/math], [math]X_n[/math] is [math]\F_n[/math]-measurable, we say that [math](X_n)_{n\geq 0}[/math] is adapted (to the filtration [math](\F_n)_{n\geq0})[/math]. One can think of [math]\F_n[/math] as the information at time [math]n[/math] and the filtration [math](\F_n)_{n\geq0}[/math] as the flow of information in time.

Let us start with a stochastic process [math](X_n)_{n\geq 0}[/math]. We define

[[math]] \F_n=\sigma(X_0,...,X_n)=\sigma(X_k\mid0\leq k\leq n). [[/math]]

By construction, [math]\mathcal{F}_n\subset\F_{n+1}[/math] and [math](X_n)_{n\geq0}[/math] is [math](\F_n)_{n\geq0}[/math]-adapted. In this case [math](\F_n)_{n\geq 0}[/math] is called the natural filtration of [math](X_n)_{n\geq 0}[/math].

In general, if [math](\F_n)_{n\geq0}[/math] is a filtration, one denotes by

[[math]] \F_\infty=\bigvee_{n\geq0}\F_n=\sigma\left(\bigcup_{n\geq 0}\F_n\right). [[/math]]

the tail [math]\sigma[/math]-Algebra.

Definition (Martingale)

Let [math](\Omega,\F,(\F_n)_{n\geq0},\p)[/math] be a filtered probability space. A stochastic process [math](X_n)_{n\geq 0}[/math] is called a martingale, if

• [math]\E[\vert X_n\vert] \lt \infty[/math] for all [math]n\geq 0[/math].
• [math]X_n[/math] is [math]\F_n[/math]-measurable (adapted).
• [math]\E[X_n\mid \F_m]=X_m[/math] a.s. for all [math]m\leq n[/math].

The last point is equivalent to say

[[math]] \E[X_{n+1}\mid\F_n]=X_na.s., [[/math]]

which can be obtained by using the tower property and induction.

Example

Let [math](X_n)_{n\geq0}[/math] be a sequence independent r.v.'s such that [math]\E[X_n]=0[/math] for all [math]n\geq 0[/math] (i.e. [math]X_n\in L^1(\Omega,\F,(\F_n)_{n\geq 0},\p)[/math]). Moreover, let [math]\F_n=\sigma(X_1,...,X_n)[/math] and [math]S_n=\sum_{i=1}^nX_i[/math] with [math]\F_0=\{\varnothing,\Omega\}[/math] and [math]S_0=0[/math]. Then [math](S_n)_{n\geq 0}[/math] is an [math]\F_n[/math]-martingale.

Show Proof

We need to check the assumptions for a martingale.

• The first point is clear by assumption on [math]X_1,...,X_n[/math] and linearity of the expectation.
  [[math]] \E[\vert S_n\vert]\leq \sum_{i=1}\E[\vert X_i\vert] \lt \infty. [[/math]]
• It is clear that [math]S_n[/math] is [math]\F_n[/math]-measurable, since it is a function [math]X_1,...,X_n[/math], which are [math]\F_n[/math]-measurable.
• Observe that
  [[math]] \begin{multline*} \E[S_{n+1}\mid \F_n]=\E[\underbrace{X_1+...+X_n}_{S_n}+X_{n+1}\mid\F_n]\\=\underbrace{\E[S_n\mid \F_n]}_{S_n}+\E[X_{n+1}\mid\F_n]=S_n+\E[X_{n+1}\mid\F_n]=S_n. \end{multline*} [[/math]]

Therefore, [math](S_n)_{n\geq0}[/math] is a martingale.

■

Example

Let [math](\Omega,\F,(\F_n)_{n\geq 0},\p)[/math] be a filtered probability space and let [math]Y\in L^1(\Omega,\F,(\F_n)_{n\geq0},\p)[/math]. Define a sequence [math](X_n)_{n\geq0}[/math] by

[[math]] X_n:=\E[Y\mid\F_n]. [[/math]]

Then [math](X_n)_{n\geq 0}[/math] is an [math]\F_n[/math]-martingale.

Show Proof

Again, we show the assumptions for a martingale.

• Since [math]\vert X_n\vert\leq \E[\vert Y\vert\mid \F_n][/math], we get
  [[math]] \E[\vert X_n\vert]\leq \E[\vert Y\vert] \lt \infty. [[/math]]
• [math]\E[Y\mid \F_n][/math] is [math]\F_n[/math]-measurable by definition.
• With the tower property, we get
  [[math]] \E[X_{n+1}\mid \F_n]=\E[\underbrace{\E[Y\mid\F_{n+1}]}_{X_{n+1}}\mid\F_n]=\E[Y\mid\F_n]=X_n. [[/math]]

Therefore, [math](X_n)_{n\geq0}[/math] is a martingale.

■

Definition (Regularity of Martingales)

Let [math](\Omega,\F,(\F_n)_{n\geq0},\p)[/math] be a filtered probability space. A martingale [math](X_n)_{n\geq 0}[/math] is said to be regular, if there exists a r.v. [math]Y\in L^1(\Omega,\F,(\F_n)_{n\geq0},\p)[/math] such that

[[math]] X_n=\E[Y\mid\F_n] [[/math]]

for all [math]n\geq 0[/math].

Proposition

Let [math](\Omega,\F,(\F_n)_{n\geq0},\p)[/math] be a filtered probability space. Let [math](X_n)_{n\geq 0}[/math] be a martingale. Then the map

[[math]] n\mapsto \E[X_n] [[/math]]

is constant, i.e. for all [math]n\geq 0[/math]

[[math]] \E[X_n]=\E[X_0]. [[/math]]

Show Proof

By the definition of a martingale, we get

[[math]] \E[X_n]=\E[\E[X_n\mid \F_0]]=X_0. [[/math]]

■

Definition (Discrete Stopping time)

Let [math](\Omega,\F,(\F_n)_{n\geq0},\p)[/math] be a filtered probability space. A r.v. [math]T:\Omega\to\bar \N=\N\cup\{\infty\}[/math] is called a stopping time if for every [math]n\geq 0[/math]

[[math]] \{T\leq n\}\in\F_n. [[/math]]

Another, more general definition is used for continuous stochastic processes and may be given in terms of a filtration. Let [math](I,\leq )[/math] be an ordered index set (often [math]I=[0,\infty)[/math]) or a compact subset thereof, thought of as the set of possible [math]times[/math]), and let [math](\Omega,\F,(\F_n)_{n\geq 0},\p)[/math] be a filtered probability space. Then a r.v. [math]T:\Omega\to I[/math] is called a stopping time if [math]\{T\leq t\}\in\F_t[/math] for all [math]t\in I[/math]. Often, to avoid confusion, we call it a [math]\F_t[/math]-stopping time and explicitly specify the filtration. Speaking concretely, for [math]T[/math] to be a stopping time, it should be possible to decide whether or not [math]\{T\leq t\}[/math] has occurred on the basis of the knowledge of [math]\F_t[/math], i.e. [math]\{T\leq t\}[/math] is [math]\F_t[/math]-measurable.

Proposition

Let [math](\Omega,\F,(\F_n)_{n\geq 0},\p)[/math] be a filtered probability space. Then

• Constant times are stopping times.
• The map
  [[math]] T:(\Omega,\F)\to(\bar\N,\mathcal{P}(\bar\N)) [[/math]]
  is a stopping time if and only if [math]\{T\leq n\}\in\F_n[/math] for all [math]n\geq 0[/math].
• If [math]S[/math] and [math]T[/math] are stopping times, then [math]S\land T[/math], [math]S\lor T[/math] and [math]S+T[/math] are also stopping times.
• Let [math](T_n)_{n\geq 0}[/math] be a sequence of stopping times. Then [math]\sup_n T[/math], [math]\inf_n T_n[/math], [math]\liminf_n T_n[/math] and [math]\limsup_n T_n[/math] are also stopping times.
• Let [math](X_n)_{n\geq 0}[/math] be a sequence of adapted r.v.'s with values in some measure space [math](E,\mathcal{E})[/math] and let [math]H\in\mathcal{E}[/math]. Then, with the convention that [math]\inf\varnothing=\infty[/math],
  [[math]] D_H=\inf\{ n\in\N\mid X_n\in H\} [[/math]]
  is a stopping time.

Show Proof

We need to show all points.

• This is clear.
• Note that
  [[math]] \{T=n\}=\{T\leq n\}\setminus \{T\leq n-1\}\in\F_n [[/math]]
  and conversely,
  [[math]] \{T\leq n\}=\bigcup_{k=0}^n\{ T=k\}\in\F_n. [[/math]]
• Just observe that
  [[math]] \{S\lor T\leq n\}=\{S\leq n\}\cap\{T\leq n\}\in\F_n [[/math]]
  [[math]] \{S\land T\leq n\}=\{S\leq n\}\cup\{T\leq n\}\in\F_n [[/math]]
  [[math]] \{S+T=n\}=\bigcup_{k=0}^n\underbrace{\{S=k\}}_{\in\F_k\subset\F_n}\cap\underbrace{\{T=n-k\}}_{\in\F_{n-k}\subset\F_n}\in\F_n [[/math]]
• First observe
  [[math]] \{\sup_{k}T_k\leq n\}=\bigcap_k\{T_k\leq n\}\in\F_n \text{and} \{\inf_k T_k\leq n\}=\bigcup_k\{T_k\leq n\}\in\F_n. [[/math]]
  Now we can rewrite
  [[math]] \limsup_k T_k=\inf_k\sup_{m\geq k}T_m \text{and} \liminf_k T_k=\sup_k\inf_{m\leq k}T_m [[/math]]
  and use the relation above.
• For all [math]n\in\N[/math], we get
  [[math]] \{D_H\leq n\}=\bigcup_{k=0}^n\underbrace{\{X_k\in H\}}_{\in\F_k\subset\F_n}\in\F_n [[/math]]

■

We say that a stopping time [math]T[/math] is bounded if there exists [math]C \gt 0[/math] such that for all [math]\omega\in\Omega[/math]

[[math]] T(\omega)\leq C [[/math]]

Without loss of generality, we can always assume that [math]C\in\N[/math]. In this case we shall denote by [math]X_T[/math], the r.v. given by

[[math]] X_T(\omega)=X_{T(\omega)}(\omega)=\sum_{n=0}^\infty X_n(\omega)\one_{\{T(\omega)=n\}}. [[/math]]

Note that the sum on the right hand side perfectly defined since [math]T(\omega)[/math] is bounded.

Theorem

Let [math](\Omega,\F,(\F_n)_{n\geq 0},\p)[/math] be a filtered probability space. Let [math]T[/math] be a bounded stopping time and let [math](X_n)_{n\geq 0}[/math] be a martingale. Then we have

[[math]] \E[X_T]=\E[X_0]. [[/math]]

Show Proof

Assume that [math]T\leq N\in\N[/math]. Then

[[math]] \begin{align*} \E[X_T]&=\E\left[\sum_{n=0}^\infty X_n\one_{\{T=n\}}\right]=\E\left[\sum_{n=0}^N X_n\one_{\{T=n\}}\right]=\sum_{n=0}^N\E[X_n\one_{\{T=n\}}]=\sum_{n=0}^N\E[\E[X_N\mid\F_n]\one_{\{T=n\}}]\\ &=\sum_{n=0}^N\E[\E[X_n\one_{\{T=n\}}\mid\F_n]]=\sum_{n=0}^N\E[X_n\one_{\{T=n\}}]=\E\left[X_n\sum_{n=0}^N\one_{\{T=n\}}\right]=\E[X_n]=\E[X_0] \end{align*} [[/math]]

■

Definition (Stopping time [math]\sigma[/math]-Algebra)

Let [math](\Omega,\F,(\F_n)_{n\geq0},\p)[/math] be a filtered probability space. Let [math]T[/math] be a stopping time for [math](\F_n)_{n\geq0}[/math]. We call the [math]\sigma[/math]-Algebra of events prior of [math]T[/math] and write [math]\F_T[/math] for the [math]\sigma[/math]-Algebra

[[math]] \F_T=\{A\in\F\mid A\cap\{T\leq n\}\in\F_n,\forall n\geq0\}. [[/math]]

We need to show that [math]\F_T[/math] is indeed a [math]\sigma[/math]-Algebra.

Proposition

If [math]T[/math] is a stopping time, [math]\F_T[/math] is a [math]\sigma[/math]-Algebra.

Show Proof

It's clear that for a filtered probability space [math](\Omega,\F,(\F_n)_{n\geq0},\p)[/math], [math]\Omega\in\F_T[/math]. If [math]A\in\F_T[/math], then

[[math]] A^C\cap \{T\leq n\}=\underbrace{\{T\leq n\}}_{\in\F_n}\setminus (\underbrace{A\cap \{T\leq n\}}_{\in\F_n})\in\F_n [[/math]]

and hence [math]A^C\in\F_n[/math]. If [math](A_i)_{i\geq0}\in\F_T[/math], then

[[math]] \bigcup_{i\geq 0}A_i\cap \{T\leq n\}=\bigcup_{i=1}^\infty A_i\cap \underbrace{\{T\leq n\}}_{\in\F_n}. [[/math]]

Hence [math]\bigcup_{i\geq 0}A_i\in\F_T[/math]. Therefore [math]\F_T[/math] is a [math]\sigma[/math]-Algebra.

■

If [math]T=n_0[/math] is constant, then [math]\F_T=\F_{n_0}[/math].

Proposition

Let [math]S[/math] and [math]T[/math] be two stopping times.

• If [math]S\leq T[/math], then [math]\F_S\subset \F_T[/math].
• [math]\F_{S\land T}=\F_S\cap\F_T[/math].
• [math]\{S\leq T\}[/math], [math]\{S=T\}[/math] and [math]\{S \lt T\}[/math] are [math]\F_S\cap\F_T[/math]-measurable.

Show Proof

We need to show all three points.

• For [math]n\in\N[/math] and [math]A\in\F_S[/math] we get
  [[math]] A\cap\{T\leq n\}= A\cap\underbrace{\{S\leq n\}\cap\{T\leq n\}}_{\{T\leq n\}}=(A\cap\{S\leq n\})\cap\underbrace{\{T\leq n\}}_{\in\F_n}\in\F_n. [[/math]]
  Therefore [math]A\in\F_T[/math].
• Since [math]S\land T\leq S[/math], we get by [math](i)[/math] that [math]\F_{S\land T}\subset\F_S[/math] and similarly that [math]\F_{S\land T}\subset\F_T[/math]. Let now [math]A\in \F_S\cap\F_T[/math]. Then
  [[math]] A\cap\{S\land T\leq n\}=\left(\underbrace{A\cap \{ S\leq n\}}_{\in\F_n,(\text{since $A\in\F_S$})}\right)\cup \left( \underbrace{A\cap\{ T\leq n\}}_{\in\F_n,(\text{since $A\in\F_T$})}\right)\in\F_n. [[/math]]
  Therefore [math]A\in\F_{S\land T}[/math].
• Note that
  [[math]] \{S\leq T\}\cap\{T=n\}=\{S\leq n\}\cap \{ T=n\}\in\F_n. [[/math]]
  Therefore [math]\{S\leq T\}\in\F_T[/math]. Note also that
  [[math]] \{S \lt T\}\cap\{T=n\}=\{S \lt n\}\cap \{T=n\}\in\F_n. [[/math]]
  Therefore [math]\{S \lt T\}\in\F_T[/math]. Finally, note that
  [[math]] \{S=T\}\cap\{T=n\}=\{S=n\}\cap\{T=n\}\in\F_n. [[/math]]
  Thus [math]\{S=T\}\in\F_T[/math]. Similarly one can show that these events are also [math]\F_S[/math]-measurable.

■

Proposition

Let [math](\Omega,\F,(\F_n)_{n\geq0},\p)[/math] be a filtered probability space. Let [math](X_n)_{n\geq 0}[/math] be a stochastic process, which is adapted, i.e. [math]X_n[/math] is [math]\F_n[/math]-measurable for all [math]n\geq 0[/math]. Let [math]T[/math] be a finite stopping time, i.e. [math]T \lt \infty[/math] a.s., such that [math]X_T[/math] is well defined. Then [math]X_T[/math] is [math]\F_T[/math]-measurable.

Show Proof

Let [math]\Lambda\in\B(\R)[/math] be a Borel measurable set. We want to show that

[[math]] \{X_T\in\Lambda\}\in\F_T, [[/math]]

that is, for all [math]n\geq 0[/math]

[[math]] \{X_T\in\Lambda\}\cap\{T\leq n\}\in\F_n. [[/math]]

Observe that

[[math]] \{X_T\in\Lambda\}\cap\{T\leq n\}=\bigcup_{k=1}^n\{X_T\in \Lambda\}\cap\{T\leq k\}=\bigcup_{k=1}^n\underbrace{\{X_k\in\Lambda\}}_{\in\F_k\subset\F_n}\cap\underbrace{\{T=k\}}_{\in\F_k\subset\F_n}, [[/math]]

which implies that [math]\{X_T\in\Lambda\}\cap\{T\leq n\}\in\F_n[/math] and the claim follows.

■

Theorem

Let [math](\Omega,\F,(\F_n)_{n\geq0},\p)[/math] be a filtered probability space. Let [math](X_n)_{n\geq 0}[/math] be a martingale and let [math]S[/math] and [math]T[/math] be two bounded stopping times such that [math]S\leq T[/math] a.s. Then we have

[[math]] \E[X_T\mid \F_S]=X_Sa.s. [[/math]]

Show Proof

Since we assume that [math]T\leq C\in\N[/math], we note that

[[math]] \vert X_T\vert\leq \sum_{i=0}^C\vert X_i\vert\in L^1(\Omega,\F,(\F_n)_{n\geq 0},\p). [[/math]]

Let now [math]A\in\F_S[/math]. We need to show that

[[math]] \E[X_T\one_A]=\E[X_S\one_A]. [[/math]]

Let us define the random time

[[math]] R(\omega)=S(\omega)\one_A(\omega)+T(\omega)\one_{A^C}(\omega). [[/math]]

We thus note that [math]R[/math] is a stopping time. Indeed, we have

[[math]] \{R\leq n\}=(\underbrace{A\cap \{S\leq n\}}_{\in\F_n})\cup (\underbrace{A^C\cap \{T\leq n\}}_{\in\F_n}). [[/math]]

Consequently, since [math]S,T[/math] and [math]R[/math] are bounded, we have

[[math]] \E[X_S]=\E[X_T]=\E[X_R]=\E[X_0]. [[/math]]

Therefore we get

[[math]] \E[X_R]=\E[X_S\one_A+X_T\one_{A^C}]\text{and}\E[X_T]=\E[X_T\one_A+X_T\one_{A^C}] [[/math]]

and thus

[[math]] \E[X_S\one_A]=\E[X_T\one_A]. [[/math]]

Moreover, since [math]X_S[/math] is [math]\F_S[/math]-measurable, we conclude that

[[math]] \E[X_T\mid \F_S]=X_Sa.s. [[/math]]

■

Proposition

Let [math](\Omega,\F,(\F_n)_{n\geq0},\p)[/math] be a filtered probability space. Let [math](X_n)_{n\geq 0}[/math] be a stochastic process such that for all [math]n\geq 0[/math]

[[math]] \E[\vert X_n\vert] \lt \infty [[/math]]

and with [math]X_n[/math] being [math]\F_n[/math]-measurable. If for all bounded stopping times [math]T[/math], we have

[[math]] \E[X_T]=\E[X_0], [[/math]]

then [math](X_n)_{n\geq 0}[/math] is a martingale.

Show Proof

Let [math]0\leq m \lt n \lt \infty[/math] and [math]\Lambda\in\F_m[/math]. Define for all [math]\omega\in\Omega[/math]

[[math]] T(\omega)=m\one_{\Lambda^C}(\omega)+n\one_\Lambda(\omega). [[/math]]

Then [math]T[/math] is a stopping time. Therefore

[[math]] \E[X_0]=\E[X_T]=\E[X_m\one_{\Lambda^C}+X_n\one_\Lambda]=\E[X_m]. [[/math]]

Hence we get

[[math]] \E[X_m\one_\Lambda]=\E[X_n\one_\Lambda] [[/math]]

and thus

[[math]] \E[X_n\mid \F_m]=X_ma.s. [[/math]]

■

General references

Moshayedi, Nima (2020). "Lectures on Probability Theory". arXiv:2010.16280 [math.PR].