Convergence in Law
We denote by [math]C_b(\R^d)[/math] the space of bounded and continuous functions [math]\varphi:\R^d\to\R[/math]. Moreover, we endow [math]C_b(\R^d)[/math] with the supremums norm [math]\|\varphi\|_\infty=\sup_{x\in\R^d}\vert\varphi(x)\vert[/math]. The space [math](C_b(\R^d),\|\cdot\|_\infty)[/math] forms a Banach space, i.e. it is a complete normed vector space. Next, we want to introduce the notion of law convergence in terms of probability measures.
- Let [math](\mu_n)_{n\geq 1}[/math] be a sequence of probability measures on [math]\R^d[/math]. We say that [math](\mu_n)_{n\geq 1}[/math] is converging weakly to a probability measure [math]\mu[/math] on [math]\R^d[/math], and we write
[[math]] \lim_{n\to\infty\atop w}\mu_n=\mu, [[/math]]if for all [math]\varphi\in C_b(\R^d)[/math] we have[[math]] \lim_{n\to\infty}\int_{\R^d} \varphi d\mu_n=\int_{\R^d} \varphi d\mu. [[/math]]
- Let [math](\Omega,\A,\p)[/math] be a probability space. A sequence of r.v.'s [math](X_n)_{n\geq 1}[/math], taking values in [math]\R^d[/math], is said to converge in law to a r.v. [math]X[/math] with values in [math]\R^d[/math] and we write
[[math]] \lim_{n\to\infty\atop law}X_n=X, [[/math]]if [math]\lim_{n\to\infty\atop w}\p_{X_n}=\p_X[/math], or equivalently if for all [math]\varphi\in C_b(\R^d)[/math] we have[[math]] \lim_{n\to\infty}\E[\varphi(X_n)]=\E[\varphi(X)]\Longleftrightarrow \lim_{n\to\infty}\int_{\R^d}\varphi(x)d\p_{X_n}(x)=\int_{\R^d}\varphi(x)d\p_{X}(x). [[/math]]
- There is an abuse of language when we say that [math]\lim_{n\to\infty\atop law}X_n=X[/math] because the r.v. [math]X[/math] is not determined in a unique way, only [math]\p_X[/math] is unique.
- Note also that the r.v.'s [math]X_n[/math] and [math]X[/math] need not be defined on the same probability space [math](\Omega,\A,\p)[/math].
- The space of probability measures on [math]\R^d[/math] can be viewed as a subspace of [math]C_b(\R^d)^*[/math] (the dual space of [math]C_b(\R^d)[/math]). The weak convergence then corresponds to convergence for the weak*-topology.
- It is enough to show that [math]\lim_{n\to\infty}\E[\varphi(X_n)]=\E[\varphi(X)][/math] or [math]\lim_{n\to\infty}\int_{\R^d}\varphi(x)d\p_{X_n}(x)=\int_{\R^d} \varphi(x)d\p_X(x)[/math] is satisfied for all [math]\varphi\in C_c(\R^d)[/math], where [math]C_c(\R^d)[/math] is the space of continuous functions with compact support. That is, [math]\varphi\in C_c(\R^d)[/math] if [math]supp(\varphi):=\overline{\{x\in\R^d\mid \varphi(x)\not=0\}}[/math] is compact.
Example
We got the following examples:
- If [math]X_n[/math] and [math]X\in\mathbb{Z}^d[/math] for all [math]n\geq 1[/math], then [math]\lim_{n\to\infty\atop law}X_n=X[/math] if and only if for all [math]X\in\mathbb{Z}^d[/math] we have
[[math]] \lim_{n\to\infty}\p[X_n=x]=\p[X=x]. [[/math]]To see this, we use point (4) of the remark above. Let therefore [math]\varphi\in C_c(\R^d)[/math]. Then[[math]] \E[\varphi(X_n)]=\sum_{k\in\mathbb{Z}^d}\varphi(k)\p[X_n=k]. [[/math]]Since [math]\varphi[/math] has compact support, i.e. [math]\varphi(x)=0[/math] for [math]\vert x\vert \leq C[/math] for some [math]C\geq 0[/math], we get[[math]] \E[\varphi(X_n)]=\sum_{k\in\mathbb{Z}^d\atop \vert k\vert\leq C}\varphi(k)\p[X_n=k]. [[/math]]Hence we have[[math]] \lim_{n\to\infty}\E[\varphi(X_n)]=\lim_{n\to\infty}\sum_{k\in\mathbb{Z}^d\atop \vert k\vert\leq C}\varphi(k)\p[X_n=k]=\sum_{k\in\mathbb{Z}^d\atop \vert k\vert \leq C}\varphi(k)\p[X=k]=\E[\varphi(X)]. [[/math]]
- If [math]X_n[/math] has density [math]\p_{X_n}(dx)=P_n(x)dx[/math] for all [math]n\geq1[/math] and if we assume that
[[math]] \lim_{n\to\infty\atop a.e.}P_n(x)=P(x), [[/math]]then there is a [math]q\geq 0[/math] such that [math]\int_{\R^d}q(x)dx \lt \infty[/math] and [math]P_n(x)\leq q(x)[/math] a.e. Then an application of the dominated convergence theorem shows that[[math]] \int_{\R^d} P(x)dx=1, [[/math]]and thus there exists a r.v. [math]X[/math] with density [math]P[/math] such that [math]\lim_{n\to\infty\atop law}X_n=X[/math] and for [math]\varphi\in C_b(\R^d)[/math] we get[[math]] \E[\varphi(X_n)]=\int_{\R^d}\varphi(x)P_n(x)dx, [[/math]]and [math]\vert\varphi(x)P_n(x)\vert\leq \underbrace{\|\varphi\|_\infty q(x)}_{\in\mathcal{L}^1(\R^d)}[/math]. So with the dominated convergence theorem we get[[math]] \lim_{n\to\infty}\int_{\R^d}\varphi(x)P_n(x)dx=\int_{\R^d}\varphi(x)P(x)dx=\E[\varphi(X)]. [[/math]]
- Let [math]X_n\sim \mathcal{N}(0,\sigma_n^2)[/math] such that [math]\lim_{n\to\infty}\sigma_n=0[/math]. Then [math]\lim_{n\to\infty\atop law}X_n=0[/math] and
[[math]] \E[\varphi(X_n)]=\int_\R \varphi(x)e^{-\frac{x^2}{2\sigma_n^2}}\frac{1}{\sigma_n\sqrt{2\pi}}dx. [[/math]]Now using that [math]u=\frac{x}{\sigma_n}[/math], we get [math]dx=\sigma_n du[/math] and hence we have[[math]] \E[\varphi(X_n)]=\int_\R \varphi(x)e^{-\frac{x^2}{2\sigma_n^2}}\frac{1}{\sigma_n\sqrt{2\pi}}dx=\int_\R \varphi(\sigma_n u)e^{-\frac{u^2}{2}}\frac{1}{\sqrt{2\pi}}du. [[/math]]Moreover, we have [math]\vert \varphi(\sigma_n u)e^{-\frac{u^2}{2}}\vert\leq \underbrace{\|\varphi\|_\infty e^{-\frac{u^2}{2}}}_{\in\mathcal{L}^1(\R)}[/math]. Hence we get[[math]] \lim_{n\to\infty}\E[\varphi(X_n)]=\int_{\R}\varphi(0)e^{-\frac{u^2}{2}}\frac{1}{\sqrt{2\pi}}du. [[/math]]
Let [math](\Omega,\A,\p)[/math] be a probability space. Let [math](X_n)_{n\geq 1}[/math] be a sequence of r.v.'s and assume that [math]\lim_{n\to\infty\atop \p}X_n=X[/math]. Then [math]\lim_{n\to\infty\atop law}X_n=X[/math].
We first note that if [math]\lim_{n\to\infty\atop a.s.}X_n=X[/math] then [math]\lim_{n\to\infty}\E[\varphi(X_n)]=\E[\varphi(X)][/math] for every [math]\varphi\in C_b(\R^d)[/math]. Let us now assume that [math](X_n)_{n\geq 1}[/math] does not converge in law to [math]X[/math]. Then there is a [math]\varphi\in C_b(\R^d)[/math] such that [math]\E[\varphi(X_n)][/math] does not converge to [math]\E[\varphi(X)][/math]. We can hence extract a subsequence [math](X_{n_k})_{k\geq 1}[/math] from [math](X_n)_{n\geq 1}[/math] and find an [math]\epsilon \gt 0[/math] such that
Let [math](\Omega,\A,\p)[/math] be a probability space. Let [math](X_n)_{n\geq 1}[/math] be a sequence of r.v.'s, A natural question would be to ask whether, under these condition, we have a [math]B\in\B(\R)[/math] such that [math]\lim_{n\to\infty}\p[X_n\in B]=\p[X\in B][/math]. If we take [math]B=\{0\}[/math] and use the previous example, we would get
which shows that the answer to the question is negative.
Let [math](\mu_n)_{n\geq 1}[/math] be a sequence of probability measures on [math]\R^d[/math] and [math]\mu[/math] be a probability measure on [math]\R^d[/math]. Then the following are equivalent.
- [math]\lim_{n\to\infty\atop w}\mu_n=\mu[/math].
- For all open subsets [math]G\subset \R^d[/math] we have
[[math]] \limsup_n\mu_n(G)\geq \mu(G). [[/math]]
- For all closed subsets [math]F\subset\R^d[/math] we have
[[math]] \limsup_n\mu_n(F)\leq \mu(F). [[/math]]
- For all Borel measurable sets [math]B\in\B(\R^d)[/math] with [math]\mu(\partial B)=0[/math] we have
[[math]] \lim_{n\to\infty}\mu_n(B)=\mu(B). [[/math]]
We immediately note that [math](ii)\Longleftrightarrow (iii)[/math] by taking complements. First we show [math](i)\Longrightarrow (ii):[/math] Let [math]G[/math] be an open subset of [math]\R^d[/math]. Define [math]\varphi_p(x):=p(d(x,G^C)\land 1)[/math]. Then [math]\varphi_p[/math] is continuous, bounded, [math]0\leq \varphi_p(x)\leq \one_{G}(x)[/math] for all [math]x\in\R^d[/math] and [math]\varphi_p\uparrow \one_{G}[/math] (note that [math]d(x,F)=\inf_{y\in F}d(x,y)[/math]) as [math]p\to\infty[/math]. Moreover, [math]F[/math] is closed if and only if [math]d(x,F)=0[/math]. We also get that [math]\varphi_p(x)=0[/math] on [math]G^C[/math] and [math]0\leq \varphi_p(x)\leq 1\leq \one_{G}(x)[/math] for all [math]x\in\R^d[/math]. Therefore we get
Consequences: We look at the case of [math]d=1[/math]. Let [math](X_n)_{n\geq 1}[/math] be a sequence of r.v.'s with values in [math]\R[/math] and let [math]X[/math] be a r.v. with values in [math]\R[/math]. One can show that
Let [math](\mu_n)_{n\geq 1}[/math] and [math]\mu[/math] be probability measures on [math]\R^d[/math]. Let [math]H\subset C_b(\R^d)[/math] such that [math]\bar H\supset C_c(\R^d)[/math]. Then the following are equivalent.
- [math]\lim_{n\to\infty\atop w}\mu_n=\mu.[/math]
- For all [math]\varphi\in C_c(\R^d)[/math] we have
[[math]] \lim_{n\to\infty}\int_{\R^d} \varphi d\mu_n=\int_{\R^d} \varphi d\mu. [[/math]]
- For all [math]\varphi\in H[/math] we have
[[math]] \lim_{n\to\infty}\int_{\R^d}\varphi d\mu_n=\int_{\R^d} \varphi d\mu. [[/math]]
It is obvious that [math](i)\Longrightarrow (ii)[/math] and [math](i)\Longrightarrow (iii)[/math]. Therefore we first show [math](ii)\Longrightarrow (i):[/math] Let therefore [math]\varphi\in C_b(\R^d)[/math] and let [math](f_k)_{k\geq 1}\in C_c(\R^d)[/math] with [math]0\leq f_k\leq 1[/math] and [math]f_k\uparrow 1[/math] as [math]k\to\infty[/math]. Then for all [math]k\geq 1[/math] we get that [math]\varphi f_k\in C_c(\R^d)[/math] and hence
Hence, for all [math]k\geq 1[/math] we get
Let [math](\Omega,\A,\p)[/math] be a probability space. Let [math](\mu_n)_{n\geq 1}[/math] be a sequence of probability measures on [math]\R^d[/math] associated to a sequence of real r.v.'s [math](X_n)_{n\geq 1}[/math]. Moreover, let [math]\hat{\mu}_n(\xi)=\int_{\R^d}e^{i\xi x}d\mu_n(x)[/math] and [math]\Phi_X(\xi)=\E[e^{i\xi x}][/math]. Then for all [math]\xi\in\R^d[/math] we get
Equivalently, for all [math]\xi\in\R^d[/math] we get
It is obvious that [math]\lim_{n\to\infty\atop w}\mu_n=\mu[/math] implies that [math]\lim_{n\to\infty}\hat{\mu}_n(\xi)=\hat{\mu}(\xi)[/math]. Therefore [math]e^{i\xi X}[/math] is continuous and bounded. For notation conventions we deal with the case [math]d=1[/math]. Let therefore [math]f\in C_c(\R^d)[/math]. For [math]\sigma \gt 0[/math] we also note [math]g_\sigma(x)=\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{x^2}{2\sigma^2}}[/math]. \begin{exer} Show that [math]g_\sigma *f\xrightarrow{\sigma\to 0}f[/math] uniformly on [math]\R[/math]. \end{exer} \begin{exer} Show that if [math]\nu[/math] is a probability measure, then
Let [math](\mu_n)_{n\geq 1}[/math] be a sequence of probability measures on [math]\R^d[/math] with characteristic functions [math](\Phi_n)_{n\geq 1}[/math]. If [math]\Phi_n[/math] converges pointwise to a function [math]\Phi[/math] which is continuous at 0, then
No proof here.
Example
Let [math](X_n)_{n\geq 1}[/math] be a sequence of poisson r.v.'s with parameter [math]\lambda[/math]. Moreover, consider the sequence [math]Z_n=\frac{X_n-1}{\sqrt{n}}[/math]. Then we have
Since [math]\E\left[e^{iu\mathcal{N}(0,1)} \right]=e^{-\frac{u^2}{2}}[/math], we deduce that [math]\lim_{n\to\infty\atop law}Z_n=\lim_{n\to\infty\atop law}\frac{X_n-u}{\sqrt{n}}=\mathcal{N}(0,1)[/math]. Before stating and proving the central limit theorem, we give two extra results on convergence in law.
Let [math](\Omega,\A,\p)[/math] be a probability space. Let [math](X_n)_{n\geq 1}[/math] be a sequence of r.v.'s and [math]X[/math] a r.v. and assume that [math]\lim_{n\to\infty\atop law}X_n=X[/math] and that [math]X[/math] is a.s. equal to a constant [math]a[/math]. Then
Let [math]f(x):=\vert x-a\vert\land 1[/math]. Then [math]f[/math] is a continuous and bounded map and therefore [math]\lim_{n\to\infty}\E[f(X_n)]=\E[f(X)]=0[/math], i.e. [math]\lim_{n\to\infty}\E[\vert X_n-a\vert \land 1]=0[/math] which implies that [math]\lim_{n\to\infty\atop \p}X_n=X[/math].
Let [math](\Omega,\A,\p)[/math] be a probability space. Let [math](X_n)_{n\geq 1}[/math] be a sequence of r.v.'s and [math]X[/math] be r.v. in [math]\R^d[/math]. Assume that [math]X_n[/math] has density [math]f_n[/math] for all [math]n\geq 1[/math] and [math]X[/math] has density [math]f[/math]. Moreover, assume that [math]\lim_{n\to\infty}f_n(x)=f(x)[/math], a.e. Then
We need to show that [math]\E[h(X_n)]\xrightarrow{n\to\infty}\E[h(X)][/math], where [math]h:\R^d\to\R[/math] is a bounded and measurable map and
General references
Moshayedi, Nima (2020). "Lectures on Probability Theory". arXiv:2010.16280 [math.PR].