guide:89f13bbb00: Difference between revisions

From Stochiki
No edit summary
 
No edit summary
Line 1: Line 1:
<div class="d-none"><math>
\newcommand{\R}{\mathbb{R}}
\newcommand{\A}{\mathcal{A}}
\newcommand{\B}{\mathcal{B}}
\newcommand{\N}{\mathbb{N}}
\newcommand{\C}{\mathbb{C}}
\newcommand{\Rbar}{\overline{\mathbb{R}}}
\newcommand{\Bbar}{\overline{\mathcal{B}}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\E}{\mathbb{E}}
\newcommand{\p}{\mathbb{P}}
\newcommand{\one}{\mathds{1}}
\newcommand{\0}{\mathcal{O}}
\newcommand{\mat}{\textnormal{Mat}}
\newcommand{\sign}{\textnormal{sign}}
\newcommand{\CP}{\mathcal{P}}
\newcommand{\CT}{\mathcal{T}}
\newcommand{\CY}{\mathcal{Y}}
\newcommand{\F}{\mathcal{F}}
\newcommand{\mathds}{\mathbb}</math></div>
We denote by <math>C_b(\R^d)</math> the space of bounded and continuous functions <math>\varphi:\R^d\to\R</math>. Moreover, we endow <math>C_b(\R^d)</math> with the supremums norm <math>\|\varphi\|_\infty=\sup_{x\in\R^d}\vert\varphi(x)\vert</math>. The space <math>(C_b(\R^d),\|\cdot\|_\infty)</math> forms a Banach space, i.e. it is a complete normed vector space. Next, we want to introduce the notion of law convergence in terms of probability measures.
{{definitioncard|Weak and Law convergence|<ul style{{=}}"list-style-type:lower-roman"><li>Let <math>(\mu_n)_{n\geq 1}</math> be a sequence of probability measures on <math>\R^d</math>. We say that <math>(\mu_n)_{n\geq 1}</math> is converging weakly to a probability measure <math>\mu</math> on <math>\R^d</math>, and we write


<math display="block">
\lim_{n\to\infty\atop w}\mu_n=\mu,
</math>
if for all <math>\varphi\in C_b(\R^d)</math> we have
<math display="block">
\lim_{n\to\infty}\int_{\R^d} \varphi d\mu_n=\int_{\R^d} \varphi d\mu.
</math>
</li>
<li>Let <math>(\Omega,\A,\p)</math> be a probability space. A sequence of r.v.'s <math>(X_n)_{n\geq 1}</math>, taking values in <math>\R^d</math>, is said to converge in law to a r.v. <math>X</math> with values in <math>\R^d</math> and we write
<math display="block">
\lim_{n\to\infty\atop law}X_n=X,
</math>
if <math>\lim_{n\to\infty\atop w}\p_{X_n}=\p_X</math>, or equivalently if for all <math>\varphi\in C_b(\R^d)</math> we have
<math display="block">
\lim_{n\to\infty}\E[\varphi(X_n)]=\E[\varphi(X)]\Longleftrightarrow \lim_{n\to\infty}\int_{\R^d}\varphi(x)d\p_{X_n}(x)=\int_{\R^d}\varphi(x)d\p_{X}(x).
</math>
</li>
</ul>}}
{{alert-info |  One has to consider the following:
<ul style{{=}}"list-style-type:lower-roman"><li>There is an abuse of language when we say that <math>\lim_{n\to\infty\atop law}X_n=X</math> because the r.v. <math>X</math> is not determined in a unique way, only <math>\p_X</math> is unique.
</li>
<li>Note also that the r.v.'s <math>X_n</math> and <math>X</math> need not be defined on the same probability space <math>(\Omega,\A,\p)</math>.
</li>
<li>The space of probability measures on <math>\R^d</math> can be viewed as a subspace of <math>C_b(\R^d)^*</math> (the dual space of <math>C_b(\R^d)</math>). The weak convergence then corresponds to convergence for the ''weak*''-topology.
</li>
<li>It is enough to show that <math>\lim_{n\to\infty}\E[\varphi(X_n)]=\E[\varphi(X)]</math> or <math>\lim_{n\to\infty}\int_{\R^d}\varphi(x)d\p_{X_n}(x)=\int_{\R^d} \varphi(x)d\p_X(x)</math> is satisfied for all <math>\varphi\in C_c(\R^d)</math>, where <math>C_c(\R^d)</math> is the space of continuous functions with compact support. That is, <math>\varphi\in C_c(\R^d)</math> if <math>supp(\varphi):=\overline{\{x\in\R^d\mid \varphi(x)\not=0\}}</math> is compact.
</li>
</ul>
}}
'''Example'''
We got the following examples:
<ul style{{=}}"list-style-type:lower-roman"><li>If <math>X_n</math> and <math>X\in\mathbb{Z}^d</math> for all <math>n\geq 1</math>, then <math>\lim_{n\to\infty\atop law}X_n=X</math> if and only if for all <math>X\in\mathbb{Z}^d</math> we have
<math display="block">
\lim_{n\to\infty}\p[X_n=x]=\p[X=x].
</math>
To see this, we use point (4) of the remark above. Let therefore <math>\varphi\in C_c(\R^d)</math>. Then
<math display="block">
\E[\varphi(X_n)]=\sum_{k\in\mathbb{Z}^d}\varphi(k)\p[X_n=k].
</math>
Since <math>\varphi</math> has compact support, i.e. <math>\varphi(x)=0</math> for <math>\vert x\vert \leq C</math> for some <math>C\geq 0</math>, we get
<math display="block">
\E[\varphi(X_n)]=\sum_{k\in\mathbb{Z}^d\atop \vert k\vert\leq C}\varphi(k)\p[X_n=k].
</math>
Hence we have
<math display="block">
\lim_{n\to\infty}\E[\varphi(X_n)]=\lim_{n\to\infty}\sum_{k\in\mathbb{Z}^d\atop \vert k\vert\leq C}\varphi(k)\p[X_n=k]=\sum_{k\in\mathbb{Z}^d\atop \vert k\vert \leq C}\varphi(k)\p[X=k]=\E[\varphi(X)].
</math>
</li>
<li>If <math>X_n</math> has density <math>\p_{X_n}(dx)=P_n(x)dx</math> for all <math>n\geq1</math> and if we assume that
<math display="block">
\lim_{n\to\infty\atop a.e.}P_n(x)=P(x),
</math>
then there is a <math>q\geq 0</math> such that <math>\int_{\R^d}q(x)dx < \infty</math> and <math>P_n(x)\leq q(x)</math> a.e. Then an application of the dominated convergence theorem shows that
<math display="block">
\int_{\R^d} P(x)dx=1,
</math>
and thus there exists a r.v. <math>X</math> with density <math>P</math> such that <math>\lim_{n\to\infty\atop law}X_n=X</math> and for <math>\varphi\in C_b(\R^d)</math> we get
<math display="block">
\E[\varphi(X_n)]=\int_{\R^d}\varphi(x)P_n(x)dx,
</math>
and <math>\vert\varphi(x)P_n(x)\vert\leq \underbrace{\|\varphi\|_\infty q(x)}_{\in\mathcal{L}^1(\R^d)}</math>. So with the dominated convergence theorem we get
<math display="block">
\lim_{n\to\infty}\int_{\R^d}\varphi(x)P_n(x)dx=\int_{\R^d}\varphi(x)P(x)dx=\E[\varphi(X)].
</math>
</li>
<li>Let <math>X_n\sim \mathcal{N}(0,\sigma_n^2)</math> such that <math>\lim_{n\to\infty}\sigma_n=0</math>. Then <math>\lim_{n\to\infty\atop law}X_n=0</math> and
<math display="block">
\E[\varphi(X_n)]=\int_\R \varphi(x)e^{-\frac{x^2}{2\sigma_n^2}}\frac{1}{\sigma_n\sqrt{2\pi}}dx.
</math>
Now using that <math>u=\frac{x}{\sigma_n}</math>, we get <math>dx=\sigma_n du</math> and hence we have
<math display="block">
\E[\varphi(X_n)]=\int_\R \varphi(x)e^{-\frac{x^2}{2\sigma_n^2}}\frac{1}{\sigma_n\sqrt{2\pi}}dx=\int_\R \varphi(\sigma_n u)e^{-\frac{u^2}{2}}\frac{1}{\sqrt{2\pi}}du.
</math>
Moreover, we have <math>\vert \varphi(\sigma_n u)e^{-\frac{u^2}{2}}\vert\leq \underbrace{\|\varphi\|_\infty e^{-\frac{u^2}{2}}}_{\in\mathcal{L}^1(\R)}</math>. Hence we get
<math display="block">
\lim_{n\to\infty}\E[\varphi(X_n)]=\int_{\R}\varphi(0)e^{-\frac{u^2}{2}}\frac{1}{\sqrt{2\pi}}du.
</math>
</li>
</ul>
{{proofcard|Proposition|prop-1|Let <math>(\Omega,\A,\p)</math> be a probability space. Let <math>(X_n)_{n\geq 1}</math> be a sequence of r.v.'s and assume that <math>\lim_{n\to\infty\atop \p}X_n=X</math>. Then <math>\lim_{n\to\infty\atop law}X_n=X</math>.
|We first note that if <math>\lim_{n\to\infty\atop a.s.}X_n=X</math> then <math>\lim_{n\to\infty}\E[\varphi(X_n)]=\E[\varphi(X)]</math> for every <math>\varphi\in C_b(\R^d)</math>. Let us now assume that <math>(X_n)_{n\geq 1}</math> does not converge in law to <math>X</math>. Then there is a <math>\varphi\in C_b(\R^d)</math> such that <math>\E[\varphi(X_n)]</math> does not converge to <math>\E[\varphi(X)]</math>. We can hence extract a subsequence <math>(X_{n_k})_{k\geq 1}</math> from <math>(X_n)_{n\geq 1}</math> and find an <math>\epsilon > 0</math> such that
<math display="block">
\vert\E[\varphi(X_{n_k})]-\E[\varphi(X)]\vert > \epsilon.
</math>
But this contradicts the fact that we can extract a further subsequence <math>(X_{n_{k_l}})_{l\geq 1}</math> from <math>(X_{n_k})_{k\geq 1}</math> such that
<math display="block">
\lim_{l\to\infty\atop a.s.} X_{n_{k_{l}}}=X.
</math>}}
Let <math>(\Omega,\A,\p)</math> be a probability space. Let <math>(X_n)_{n\geq 1}</math> be a sequence of r.v.'s, A natural question would be to ask whether, under these condition, we have a <math>B\in\B(\R)</math> such that <math>\lim_{n\to\infty}\p[X_n\in B]=\p[X\in B]</math>. If we take <math>B=\{0\}</math> and use the previous example, we would get
<math display="block">
\lim_{n\to\infty}\underbrace{\p[X_n=0]}_{=0}\not=\underbrace{\p[X=0]}_{=1},
</math>
which shows that the answer to the question is negative.
{{proofcard|Proposition|prop-2|Let <math>(\mu_n)_{n\geq 1}</math> be a sequence of probability measures on <math>\R^d</math> and <math>\mu</math> be a probability measure on <math>\R^d</math>. Then the following are equivalent.
<ul style{{=}}"list-style-type:lower-roman"><li><math>\lim_{n\to\infty\atop w}\mu_n=\mu</math>.
</li>
<li>For all open subsets <math>G\subset \R^d</math> we have
<math display="block">
\limsup_n\mu_n(G)\geq \mu(G).
</math>
</li>
<li>For all closed subsets <math>F\subset\R^d</math> we have
<math display="block">
\limsup_n\mu_n(F)\leq \mu(F).
</math>
</li>
<li>For all Borel measurable sets <math>B\in\B(\R^d)</math> with <math>\mu(\partial B)=0</math> we have
<math display="block">
\lim_{n\to\infty}\mu_n(B)=\mu(B).
</math>
</li>
</ul>
|We immediately note that <math>(ii)\Longleftrightarrow (iii)</math> by taking complements.
First we show <math>(i)\Longrightarrow (ii):</math> Let <math>G</math> be an open subset of <math>\R^d</math>. Define <math>\varphi_p(x):=p(d(x,G^C)\land 1)</math>. Then <math>\varphi_p</math> is continuous, bounded, <math>0\leq \varphi_p(x)\leq \one_{G}(x)</math> for all <math>x\in\R^d</math> and <math>\varphi_p\uparrow \one_{G}</math> (note that <math>d(x,F)=\inf_{y\in F}d(x,y)</math>) as <math>p\to\infty</math>. Moreover, <math>F</math> is closed if and only if <math>d(x,F)=0</math>. We also get that <math>\varphi_p(x)=0</math> on <math>G^C</math> and <math>0\leq \varphi_p(x)\leq 1\leq \one_{G}(x)</math> for all <math>x\in\R^d</math>. Therefore we get
<math display="block">
\liminf_n\mu_n(G)\geq \sup_{p}\left(\liminf_nF\int_{\R^d}\varphi_pd\mu_n\right)=\sup_p\int_{\R^d} \varphi_pd\mu=\int \one_G d\mu=\mu(G).
</math>
Now we show that <math>(ii)</math> and <math>(iii)\Longrightarrow (iv):</math> For Borel measurable set <math>B\in\B(\R^d)</math> with <math>\mathring{B}\subset B\subset \bar B</math> we get
<math display="block">
\limsup_n\mu_n(B)\leq \limsup_n\mu_n(\bar B)\leq \mu(\bar B),
</math>
<math display="block">
\liminf_n\mu_n(B)\geq \liminf_n\mu_n(\mathring{B})\geq \mu(\mathring{B}).
</math>
Therefore it follows that
<math display="block">
\mu(\mathring{B})\leq \liminf_n \mu_n(B)\leq \limsup_n\mu_n(B)\leq \mu(B).
</math>
Moreover, if <math>\mu(\partial B)=0</math>, we get that <math>\mu(\bar B)=\mu(\mathring{B})=\mu(B)</math> and thus <math>\lim_{n\to\infty}\mu_n(B)=\mu(B)</math>.
Now we show <math>(iv)\Longrightarrow (i):</math> Let therefore <math>\varphi\in C_b(\R^d)</math>. We can always use that <math>\varphi=\varphi^+-\varphi^-</math> and so, without loss of generality, we may assume that <math>\varphi\geq 0</math>. Let <math>\varphi\geq 0</math> and <math>K\geq 0</math> be such that <math>0\leq \varphi\leq K</math>. Then
<math display="block">
\int_{\R^d}\varphi(x)d\mu(x)=\int_{\R^d}\underbrace{\left(\int_0^K\one_{\{t\leq \varphi(x)\}}dt\right)}_{K\land \varphi(x)=\varphi(x)}d\mu(x)=\int_0^K\mu(E_t^\varphi)dt,
</math>
where <math>E_t^\varphi:=\{x\in\R^d\mid \varphi(x)\geq t\}</math>. Similarly, we have
<math display="block">
\int_{\R^d}\varphi(x)d\mu_n(x)=\int_0^K\mu_n(E_t^\varphi)dt.
</math>
Now we can note that <math>\partial E_t^\varphi\subset\{x\in\R^d\mid \varphi(x)=t\}</math>. Moreover, there are at most countably many values of <math>t</math> for which <math>\mu\left(\{x\in\R^d\mid \varphi(x)=t\}\right) > 0</math>. Indeed, for an integer<math>K\geq 1</math> we get that <math>\mu\left(\{x\in\R^d\mid \varphi(x)=t\}\right)\geq \frac{1}{K}</math>. This can happen for at most <math>K</math> distinct values of <math>t</math>. Thence we have
<math display="block">
\lim_{n\to\infty}\mu_n(E_t^\varphi)=\mu(E_t^\varphi)dt\text{a.e.},
</math>
which implies that
<math display="block">
\lim_{n\to\infty}\int_{\R^d} \varphi(x)d\mu_n(x)=\int_0^K\mu_n(E_t^\varphi)dt\xrightarrow{n\to\infty} \int_0^K\mu(E_t^\varphi)dt=\int_{\R^d}\varphi(x)d\mu(x).
</math>}}
''Consequences:'' We look at the case of <math>d=1</math>. Let <math>(X_n)_{n\geq 1}</math> be a sequence of r.v.'s with values in <math>\R</math> and let <math>X</math> be a r.v. with values in <math>\R</math>. One can show that
<math display="block">
\lim_{n\to\infty\atop law}X_n=X\Longleftrightarrow \lim_{n\to\infty}F_{X_n}(t)=F_X(t).
</math>
{{proofcard|Proposition|prop-3|Let <math>(\mu_n)_{n\geq 1}</math> and <math>\mu</math> be probability measures on <math>\R^d</math>. Let <math>H\subset C_b(\R^d)</math> such that <math>\bar H\supset C_c(\R^d)</math>. Then the following are equivalent.
<ul style{{=}}"list-style-type:lower-roman"><li><math>\lim_{n\to\infty\atop w}\mu_n=\mu.</math>
</li>
<li>For all <math>\varphi\in C_c(\R^d)</math> we have
<math display="block">
\lim_{n\to\infty}\int_{\R^d} \varphi d\mu_n=\int_{\R^d} \varphi d\mu.
</math>
</li>
<li>For all <math>\varphi\in H</math> we have
<math display="block">
\lim_{n\to\infty}\int_{\R^d}\varphi d\mu_n=\int_{\R^d} \varphi d\mu.
</math>
</li>
</ul>
|It is obvious that <math>(i)\Longrightarrow (ii)</math> and <math>(i)\Longrightarrow (iii)</math>.
Therefore we first show <math>(ii)\Longrightarrow (i):</math> Let therefore <math>\varphi\in C_b(\R^d)</math> and let <math>(f_k)_{k\geq 1}\in C_c(\R^d)</math> with <math>0\leq f_k\leq 1</math> and <math>f_k\uparrow 1</math> as <math>k\to\infty</math>. Then for all <math>k\geq 1</math> we get that <math>\varphi f_k\in C_c(\R^d)</math> and hence
<math display="block">
\lim_{n\to\infty}\int_{\R^d}\varphi f_kd\mu_n=\int_{\R^d} \varphi f_k d\mu.
</math>
Moreover, we have
<math display="block">
\left\vert \int_{\R^d} \varphi d\mu-\int_{\R^d}\varphi f_k d\mu\right\vert\leq \sup_{x\in\R^d}\vert\varphi(x)\vert\left(1-\int_{\R^d} f_kd\mu\right)
</math>
and also
<math display="block">
\left\vert \int_{\R^d} \varphi d\mu_n-\int_{\R^d} \varphi f_k d\mu_n\right\vert\leq \sup_{x\in\R^d}\vert \varphi(x)\vert \left(1-\int_{\R^d} f_k d\mu_n\right).
</math>
Hence, for all <math>k\geq 1</math> we get
<math display="block">
\begin{align*}
\limsup_n \left\vert\int_{\R^d}\varphi d\mu-\int_{\R^d} \varphi  d\mu_n\right\vert&\leq \sup_{x\in \R^d}\vert \varphi(x)\vert \limsup_n\left[\left(1-\int_{\R^d} f_k d\mu_n\right)+\left(1-\int_{\R^d} f_k d\mu\right)\right]\\
&=2\sup_{x\in\R^d}\vert\varphi(x)\vert\left(1-\int f_kd\mu\right)\xrightarrow{k\to\infty}0.
\end{align*}
</math>
Now we show <math>(iii)\Longrightarrow (ii):</math> Let therefore <math>\varphi\in C_c(\R^d)</math>. Then there is a sequence <math>(\varphi_k)_{k\geq 1}\subset H</math> such that <math>\|\varphi-\varphi_k\|_\infty\leq \frac{1}{k}</math> for all <math>k\geq 1</math>. This implies that
<math display="block">
\begin{multline*}
\limsup_n\left\vert\int \varphi d\mu_n-\int \varphi d\mu\right\vert\\\leq \limsup_n\left(\left\vert\int \varphi d\mu_n-\int\varphi_k d\mu_n\right\vert+\underbrace{\left\vert\int \varphi_k d\mu_n-\int\varphi_k d\mu\right\vert}_{\xrightarrow{n\to\infty}0} +\left\vert \int \varphi_k d\mu-\int\varphi d\mu\right\vert\right)\leq \frac{2}{k}.
\end{multline*}
</math>
The claim follows now for <math>k\to\infty</math>.}}
{{proofcard|Theorem (Lèvy)|thm-1|Let <math>(\Omega,\A,\p)</math> be a probability space. Let <math>(\mu_n)_{n\geq 1}</math> be a sequence of probability measures on <math>\R^d</math> associated to a sequence of real r.v.'s <math>(X_n)_{n\geq 1}</math>. Moreover, let <math>\hat{\mu}_n(\xi)=\int_{\R^d}e^{i\xi x}d\mu_n(x)</math> and <math>\Phi_X(\xi)=\E[e^{i\xi x}]</math>. Then for all <math>\xi\in\R^d</math> we get
<math display="block">
\lim_{n\to\infty\atop w}\mu_n=\mu \Longleftrightarrow\lim_{n\to\infty}\hat\mu_n(\xi)=\hat\mu(\xi).
</math>
Equivalently, for all <math>\xi\in\R^d</math> we get
<math display="block">
\lim_{n\to\infty\atop law}X_n=X\Longleftrightarrow\lim_{n\to\infty}\Phi_{X_n}(\xi)=\Phi_X(\xi).
</math>
|It is obvious that <math>\lim_{n\to\infty\atop w}\mu_n=\mu</math> implies that <math>\lim_{n\to\infty}\hat{\mu}_n(\xi)=\hat{\mu}(\xi)</math>. Therefore <math>e^{i\xi X}</math> is continuous and bounded. For notation conventions we deal with the case <math>d=1</math>. Let therefore <math>f\in C_c(\R^d)</math>. For <math>\sigma > 0</math> we also note <math>g_\sigma(x)=\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{x^2}{2\sigma^2}}</math>.
\begin{exer}
Show that <math>g_\sigma *f\xrightarrow{\sigma\to 0}f</math> uniformly on <math>\R</math>.
\end{exer}
\begin{exer}
Show that if <math>\nu</math> is a probability measure, then
<math display="block">
\int_\R g_\sigma * fd\nu=\int_\R f(x)(g_\sigma *\nu)(x)dx=\int_\R f(x)\frac{1}{\sigma\sqrt{2\pi}}\int_\R e^{i\xi x}g_{\frac{1}{\sigma}}(\xi)\hat{\nu}(\xi)d\xi dx.
</math>
\end{exer}
Since <math>\lim_{n\to\infty}\hat{\mu}_n(\xi)=\hat{\mu}(\xi)</math>, we get by the dominated convergence theorem that
<math display="block">
\int_\R e^{i\xi x}g_{\frac{1}{\sigma}}(\xi)\hat{\mu}_n(\xi)d\xi\xrightarrow{n\to\infty}\int_\R e^{i\xi x}g_{\frac{1}{\sigma}}(\xi)\hat{\mu}(\xi)d\xi.
</math>
These quantities are bounded by 1, and hence we can apply the dominated convergence theorem to obtain
<math display="block">
\int_\R g_\sigma * fd\mu_n\xrightarrow{n\to\infty} \int_\R g_\sigma * fd\mu.
</math>
Let now <math>H:=\{\varphi=g_\sigma *f\mid \sigma > 0,f\in C_c(\R^d)\}\subset C_b(\R^d)</math>. Since <math>f\in C_c(\R^d)</math> we get that <math>\|g_\sigma *f-f\|_{\infty}\xrightarrow{n\to\infty}0</math> and thus <math>\bar H\supset C_c(\R^d)</math>. The result now follows from the previous proposition.}}
{{proofcard|Theorem (Lévy)|thm-2|Let <math>(\mu_n)_{n\geq 1}</math> be a sequence of probability measures on <math>\R^d</math> with characteristic functions <math>(\Phi_n)_{n\geq 1}</math>. If <math>\Phi_n</math> converges pointwise to a function <math>\Phi</math> which is continuous at 0, then
<math display="block">
\lim_{n\to\infty\atop w}\mu_n=\mu
</math>
for some probability measure <math>\mu</math> on <math>\R^d</math>.
|No proof here.}}
'''Example'''
Let <math>(X_n)_{n\geq 1}</math> be a sequence of poisson r.v.'s with parameter <math>\lambda</math>. Moreover, consider the sequence <math>Z_n=\frac{X_n-1}{\sqrt{n}}</math>. Then we have
<math display="block">
\begin{multline*}
E\left[e^{iu Z_n}\right]=\E\left[e^{iu\left(\frac{X_n-u}{\sqrt{n}}\right)}\right]=e^{-i\frac{u}{\sqrt{n}}}\E\left[e^{iu\frac{X_n}{\sqrt{n}}}\right]=e^{-i\frac{u}{\sqrt{n}}}e^{\lambda\left(e^{i \frac{u}{\sqrt{n}}}-1\right)}\\=e^{-i\frac{u}{\sqrt{n}}}e^{\lambda\left(i\frac{u}{\sqrt{n}}-\frac{u^2}{2n}+O\left(\frac{1}{n}\right)\right)}\xrightarrow{n\to\infty}e^{-\frac{u^2}{2}}.
\end{multline*}
</math>
Since <math>\E\left[e^{iu\mathcal{N}(0,1)} \right]=e^{-\frac{u^2}{2}}</math>, we deduce that <math>\lim_{n\to\infty\atop law}Z_n=\lim_{n\to\infty\atop law}\frac{X_n-u}{\sqrt{n}}=\mathcal{N}(0,1)</math>. Before stating and proving the central limit theorem, we give two extra results on convergence in law.
{{proofcard|Theorem|thm-3|Let <math>(\Omega,\A,\p)</math> be a probability space. Let <math>(X_n)_{n\geq 1}</math> be a sequence of r.v.'s and <math>X</math> a r.v. and assume that <math>\lim_{n\to\infty\atop law}X_n=X</math> and that <math>X</math> is a.s. equal to a constant <math>a</math>. Then
<math display="block">
\lim_{n\to\infty\atop\p}X_n=X.
</math>
|Let <math>f(x):=\vert x-a\vert\land 1</math>. Then <math>f</math> is a continuous and bounded map and therefore <math>\lim_{n\to\infty}\E[f(X_n)]=\E[f(X)]=0</math>, i.e. <math>\lim_{n\to\infty}\E[\vert X_n-a\vert \land 1]=0</math> which implies that <math>\lim_{n\to\infty\atop \p}X_n=X</math>.}}
{{proofcard|Theorem|thm-4|Let <math>(\Omega,\A,\p)</math> be a probability space. Let <math>(X_n)_{n\geq 1}</math> be a sequence of r.v.'s and <math>X</math> be r.v. in <math>\R^d</math>. Assume that <math>X_n</math> has density <math>f_n</math> for all <math>n\geq 1</math> and <math>X</math> has density <math>f</math>. Moreover, assume that <math>\lim_{n\to\infty}f_n(x)=f(x)</math>, a.e. Then
<math display="block">
\lim_{n\to\infty\atop law}X_n=X.
</math>
|We need to show that <math>\E[h(X_n)]\xrightarrow{n\to\infty}\E[h(X)]</math>, where <math>h:\R^d\to\R</math> is a bounded and measurable map and
<math display="block">
\E[h(X_n)]=\int_{\R^d}h(x)f_n(x)dx,
</math>
<math display="block">
\E[h(X)]=\int_{\R^d}h(x)f(x)dx.
</math>
Let <math>h:\R^d\to\R</math> be a bounded and measurable map. Moreover, let <math>\alpha=\sup_{x\in\R^d}\vert h(x)\vert.</math> Set <math>h_1(x)=h(x)+\alpha\geq 0</math> and <math>h_2(x)=\alpha-h(x)\geq 0</math>. So it follows that <math>h_1f_n\geq 0</math> and <math>h_2f_n\geq 0</math>. Moreover, we also get that
<math display="block">
h_1f_n\xrightarrow{n\to\infty}h_1f\text{a.e.}
</math>
<math display="block">
h_2f_n\xrightarrow{n\to\infty}h_2f\text{a.e.}
</math>
With Fatou's lemma we get
<math display="block">
\E[h_1(X)]=\int_{\R^d}h_1(x)f(x)dx\leq \liminf_n\int_{\R^d}h_1(x)f_n(x)dx=\liminf_n\E[h_1(X_n)].
</math>
Similarly we get <math>\E[h_2(X)]\leq \liminf_n\E[h_2(X_n)]</math>. Now substitute <math>h_1(x)=h(x)+\alpha</math> and <math>h_2(x)=\alpha-h(x)</math>, where we use <math>\liminf_n(-a_n)=\liminf_n(a_n)</math> to obtain
<math display="block">
\limsup_n\E[h(X_n)]\leq \E[h(X)]\leq \liminf_n\E[h(X_n)],
</math>
which implies that
<math display="block">
\liminf_n\E[h(X_n)]=\limsup_n\E[h(X_n)]=\E[h(X)].
</math>}}
==General references==
{{cite arXiv|last=Moshayedi|first=Nima|year=2020|title=Lectures on Probability Theory|eprint=2010.16280|class=math.PR}}

Revision as of 01:53, 8 May 2024

[math] \newcommand{\R}{\mathbb{R}} \newcommand{\A}{\mathcal{A}} \newcommand{\B}{\mathcal{B}} \newcommand{\N}{\mathbb{N}} \newcommand{\C}{\mathbb{C}} \newcommand{\Rbar}{\overline{\mathbb{R}}} \newcommand{\Bbar}{\overline{\mathcal{B}}} \newcommand{\Q}{\mathbb{Q}} \newcommand{\E}{\mathbb{E}} \newcommand{\p}{\mathbb{P}} \newcommand{\one}{\mathds{1}} \newcommand{\0}{\mathcal{O}} \newcommand{\mat}{\textnormal{Mat}} \newcommand{\sign}{\textnormal{sign}} \newcommand{\CP}{\mathcal{P}} \newcommand{\CT}{\mathcal{T}} \newcommand{\CY}{\mathcal{Y}} \newcommand{\F}{\mathcal{F}} \newcommand{\mathds}{\mathbb}[/math]

We denote by [math]C_b(\R^d)[/math] the space of bounded and continuous functions [math]\varphi:\R^d\to\R[/math]. Moreover, we endow [math]C_b(\R^d)[/math] with the supremums norm [math]\|\varphi\|_\infty=\sup_{x\in\R^d}\vert\varphi(x)\vert[/math]. The space [math](C_b(\R^d),\|\cdot\|_\infty)[/math] forms a Banach space, i.e. it is a complete normed vector space. Next, we want to introduce the notion of law convergence in terms of probability measures.

Definition (Weak and Law convergence)

  • Let [math](\mu_n)_{n\geq 1}[/math] be a sequence of probability measures on [math]\R^d[/math]. We say that [math](\mu_n)_{n\geq 1}[/math] is converging weakly to a probability measure [math]\mu[/math] on [math]\R^d[/math], and we write
    [[math]] \lim_{n\to\infty\atop w}\mu_n=\mu, [[/math]]
    if for all [math]\varphi\in C_b(\R^d)[/math] we have
    [[math]] \lim_{n\to\infty}\int_{\R^d} \varphi d\mu_n=\int_{\R^d} \varphi d\mu. [[/math]]
  • Let [math](\Omega,\A,\p)[/math] be a probability space. A sequence of r.v.'s [math](X_n)_{n\geq 1}[/math], taking values in [math]\R^d[/math], is said to converge in law to a r.v. [math]X[/math] with values in [math]\R^d[/math] and we write
    [[math]] \lim_{n\to\infty\atop law}X_n=X, [[/math]]
    if [math]\lim_{n\to\infty\atop w}\p_{X_n}=\p_X[/math], or equivalently if for all [math]\varphi\in C_b(\R^d)[/math] we have
    [[math]] \lim_{n\to\infty}\E[\varphi(X_n)]=\E[\varphi(X)]\Longleftrightarrow \lim_{n\to\infty}\int_{\R^d}\varphi(x)d\p_{X_n}(x)=\int_{\R^d}\varphi(x)d\p_{X}(x). [[/math]]

One has to consider the following:
  • There is an abuse of language when we say that [math]\lim_{n\to\infty\atop law}X_n=X[/math] because the r.v. [math]X[/math] is not determined in a unique way, only [math]\p_X[/math] is unique.
  • Note also that the r.v.'s [math]X_n[/math] and [math]X[/math] need not be defined on the same probability space [math](\Omega,\A,\p)[/math].
  • The space of probability measures on [math]\R^d[/math] can be viewed as a subspace of [math]C_b(\R^d)^*[/math] (the dual space of [math]C_b(\R^d)[/math]). The weak convergence then corresponds to convergence for the weak*-topology.
  • It is enough to show that [math]\lim_{n\to\infty}\E[\varphi(X_n)]=\E[\varphi(X)][/math] or [math]\lim_{n\to\infty}\int_{\R^d}\varphi(x)d\p_{X_n}(x)=\int_{\R^d} \varphi(x)d\p_X(x)[/math] is satisfied for all [math]\varphi\in C_c(\R^d)[/math], where [math]C_c(\R^d)[/math] is the space of continuous functions with compact support. That is, [math]\varphi\in C_c(\R^d)[/math] if [math]supp(\varphi):=\overline{\{x\in\R^d\mid \varphi(x)\not=0\}}[/math] is compact.

Example

We got the following examples:

  • If [math]X_n[/math] and [math]X\in\mathbb{Z}^d[/math] for all [math]n\geq 1[/math], then [math]\lim_{n\to\infty\atop law}X_n=X[/math] if and only if for all [math]X\in\mathbb{Z}^d[/math] we have
    [[math]] \lim_{n\to\infty}\p[X_n=x]=\p[X=x]. [[/math]]
    To see this, we use point (4) of the remark above. Let therefore [math]\varphi\in C_c(\R^d)[/math]. Then
    [[math]] \E[\varphi(X_n)]=\sum_{k\in\mathbb{Z}^d}\varphi(k)\p[X_n=k]. [[/math]]
    Since [math]\varphi[/math] has compact support, i.e. [math]\varphi(x)=0[/math] for [math]\vert x\vert \leq C[/math] for some [math]C\geq 0[/math], we get
    [[math]] \E[\varphi(X_n)]=\sum_{k\in\mathbb{Z}^d\atop \vert k\vert\leq C}\varphi(k)\p[X_n=k]. [[/math]]
    Hence we have
    [[math]] \lim_{n\to\infty}\E[\varphi(X_n)]=\lim_{n\to\infty}\sum_{k\in\mathbb{Z}^d\atop \vert k\vert\leq C}\varphi(k)\p[X_n=k]=\sum_{k\in\mathbb{Z}^d\atop \vert k\vert \leq C}\varphi(k)\p[X=k]=\E[\varphi(X)]. [[/math]]
  • If [math]X_n[/math] has density [math]\p_{X_n}(dx)=P_n(x)dx[/math] for all [math]n\geq1[/math] and if we assume that
    [[math]] \lim_{n\to\infty\atop a.e.}P_n(x)=P(x), [[/math]]
    then there is a [math]q\geq 0[/math] such that [math]\int_{\R^d}q(x)dx \lt \infty[/math] and [math]P_n(x)\leq q(x)[/math] a.e. Then an application of the dominated convergence theorem shows that
    [[math]] \int_{\R^d} P(x)dx=1, [[/math]]
    and thus there exists a r.v. [math]X[/math] with density [math]P[/math] such that [math]\lim_{n\to\infty\atop law}X_n=X[/math] and for [math]\varphi\in C_b(\R^d)[/math] we get
    [[math]] \E[\varphi(X_n)]=\int_{\R^d}\varphi(x)P_n(x)dx, [[/math]]
    and [math]\vert\varphi(x)P_n(x)\vert\leq \underbrace{\|\varphi\|_\infty q(x)}_{\in\mathcal{L}^1(\R^d)}[/math]. So with the dominated convergence theorem we get
    [[math]] \lim_{n\to\infty}\int_{\R^d}\varphi(x)P_n(x)dx=\int_{\R^d}\varphi(x)P(x)dx=\E[\varphi(X)]. [[/math]]
  • Let [math]X_n\sim \mathcal{N}(0,\sigma_n^2)[/math] such that [math]\lim_{n\to\infty}\sigma_n=0[/math]. Then [math]\lim_{n\to\infty\atop law}X_n=0[/math] and
    [[math]] \E[\varphi(X_n)]=\int_\R \varphi(x)e^{-\frac{x^2}{2\sigma_n^2}}\frac{1}{\sigma_n\sqrt{2\pi}}dx. [[/math]]
    Now using that [math]u=\frac{x}{\sigma_n}[/math], we get [math]dx=\sigma_n du[/math] and hence we have
    [[math]] \E[\varphi(X_n)]=\int_\R \varphi(x)e^{-\frac{x^2}{2\sigma_n^2}}\frac{1}{\sigma_n\sqrt{2\pi}}dx=\int_\R \varphi(\sigma_n u)e^{-\frac{u^2}{2}}\frac{1}{\sqrt{2\pi}}du. [[/math]]
    Moreover, we have [math]\vert \varphi(\sigma_n u)e^{-\frac{u^2}{2}}\vert\leq \underbrace{\|\varphi\|_\infty e^{-\frac{u^2}{2}}}_{\in\mathcal{L}^1(\R)}[/math]. Hence we get
    [[math]] \lim_{n\to\infty}\E[\varphi(X_n)]=\int_{\R}\varphi(0)e^{-\frac{u^2}{2}}\frac{1}{\sqrt{2\pi}}du. [[/math]]
Proposition

Let [math](\Omega,\A,\p)[/math] be a probability space. Let [math](X_n)_{n\geq 1}[/math] be a sequence of r.v.'s and assume that [math]\lim_{n\to\infty\atop \p}X_n=X[/math]. Then [math]\lim_{n\to\infty\atop law}X_n=X[/math].


Show Proof

We first note that if [math]\lim_{n\to\infty\atop a.s.}X_n=X[/math] then [math]\lim_{n\to\infty}\E[\varphi(X_n)]=\E[\varphi(X)][/math] for every [math]\varphi\in C_b(\R^d)[/math]. Let us now assume that [math](X_n)_{n\geq 1}[/math] does not converge in law to [math]X[/math]. Then there is a [math]\varphi\in C_b(\R^d)[/math] such that [math]\E[\varphi(X_n)][/math] does not converge to [math]\E[\varphi(X)][/math]. We can hence extract a subsequence [math](X_{n_k})_{k\geq 1}[/math] from [math](X_n)_{n\geq 1}[/math] and find an [math]\epsilon \gt 0[/math] such that

[[math]] \vert\E[\varphi(X_{n_k})]-\E[\varphi(X)]\vert \gt \epsilon. [[/math]]
But this contradicts the fact that we can extract a further subsequence [math](X_{n_{k_l}})_{l\geq 1}[/math] from [math](X_{n_k})_{k\geq 1}[/math] such that

[[math]] \lim_{l\to\infty\atop a.s.} X_{n_{k_{l}}}=X. [[/math]]

Let [math](\Omega,\A,\p)[/math] be a probability space. Let [math](X_n)_{n\geq 1}[/math] be a sequence of r.v.'s, A natural question would be to ask whether, under these condition, we have a [math]B\in\B(\R)[/math] such that [math]\lim_{n\to\infty}\p[X_n\in B]=\p[X\in B][/math]. If we take [math]B=\{0\}[/math] and use the previous example, we would get

[[math]] \lim_{n\to\infty}\underbrace{\p[X_n=0]}_{=0}\not=\underbrace{\p[X=0]}_{=1}, [[/math]]

which shows that the answer to the question is negative.

Proposition

Let [math](\mu_n)_{n\geq 1}[/math] be a sequence of probability measures on [math]\R^d[/math] and [math]\mu[/math] be a probability measure on [math]\R^d[/math]. Then the following are equivalent.

  • [math]\lim_{n\to\infty\atop w}\mu_n=\mu[/math].
  • For all open subsets [math]G\subset \R^d[/math] we have
    [[math]] \limsup_n\mu_n(G)\geq \mu(G). [[/math]]
  • For all closed subsets [math]F\subset\R^d[/math] we have
    [[math]] \limsup_n\mu_n(F)\leq \mu(F). [[/math]]
  • For all Borel measurable sets [math]B\in\B(\R^d)[/math] with [math]\mu(\partial B)=0[/math] we have
    [[math]] \lim_{n\to\infty}\mu_n(B)=\mu(B). [[/math]]


Show Proof

We immediately note that [math](ii)\Longleftrightarrow (iii)[/math] by taking complements. First we show [math](i)\Longrightarrow (ii):[/math] Let [math]G[/math] be an open subset of [math]\R^d[/math]. Define [math]\varphi_p(x):=p(d(x,G^C)\land 1)[/math]. Then [math]\varphi_p[/math] is continuous, bounded, [math]0\leq \varphi_p(x)\leq \one_{G}(x)[/math] for all [math]x\in\R^d[/math] and [math]\varphi_p\uparrow \one_{G}[/math] (note that [math]d(x,F)=\inf_{y\in F}d(x,y)[/math]) as [math]p\to\infty[/math]. Moreover, [math]F[/math] is closed if and only if [math]d(x,F)=0[/math]. We also get that [math]\varphi_p(x)=0[/math] on [math]G^C[/math] and [math]0\leq \varphi_p(x)\leq 1\leq \one_{G}(x)[/math] for all [math]x\in\R^d[/math]. Therefore we get

[[math]] \liminf_n\mu_n(G)\geq \sup_{p}\left(\liminf_nF\int_{\R^d}\varphi_pd\mu_n\right)=\sup_p\int_{\R^d} \varphi_pd\mu=\int \one_G d\mu=\mu(G). [[/math]]
Now we show that [math](ii)[/math] and [math](iii)\Longrightarrow (iv):[/math] For Borel measurable set [math]B\in\B(\R^d)[/math] with [math]\mathring{B}\subset B\subset \bar B[/math] we get

[[math]] \limsup_n\mu_n(B)\leq \limsup_n\mu_n(\bar B)\leq \mu(\bar B), [[/math]]

[[math]] \liminf_n\mu_n(B)\geq \liminf_n\mu_n(\mathring{B})\geq \mu(\mathring{B}). [[/math]]
Therefore it follows that

[[math]] \mu(\mathring{B})\leq \liminf_n \mu_n(B)\leq \limsup_n\mu_n(B)\leq \mu(B). [[/math]]
Moreover, if [math]\mu(\partial B)=0[/math], we get that [math]\mu(\bar B)=\mu(\mathring{B})=\mu(B)[/math] and thus [math]\lim_{n\to\infty}\mu_n(B)=\mu(B)[/math]. Now we show [math](iv)\Longrightarrow (i):[/math] Let therefore [math]\varphi\in C_b(\R^d)[/math]. We can always use that [math]\varphi=\varphi^+-\varphi^-[/math] and so, without loss of generality, we may assume that [math]\varphi\geq 0[/math]. Let [math]\varphi\geq 0[/math] and [math]K\geq 0[/math] be such that [math]0\leq \varphi\leq K[/math]. Then

[[math]] \int_{\R^d}\varphi(x)d\mu(x)=\int_{\R^d}\underbrace{\left(\int_0^K\one_{\{t\leq \varphi(x)\}}dt\right)}_{K\land \varphi(x)=\varphi(x)}d\mu(x)=\int_0^K\mu(E_t^\varphi)dt, [[/math]]
where [math]E_t^\varphi:=\{x\in\R^d\mid \varphi(x)\geq t\}[/math]. Similarly, we have

[[math]] \int_{\R^d}\varphi(x)d\mu_n(x)=\int_0^K\mu_n(E_t^\varphi)dt. [[/math]]
Now we can note that [math]\partial E_t^\varphi\subset\{x\in\R^d\mid \varphi(x)=t\}[/math]. Moreover, there are at most countably many values of [math]t[/math] for which [math]\mu\left(\{x\in\R^d\mid \varphi(x)=t\}\right) \gt 0[/math]. Indeed, for an integer[math]K\geq 1[/math] we get that [math]\mu\left(\{x\in\R^d\mid \varphi(x)=t\}\right)\geq \frac{1}{K}[/math]. This can happen for at most [math]K[/math] distinct values of [math]t[/math]. Thence we have

[[math]] \lim_{n\to\infty}\mu_n(E_t^\varphi)=\mu(E_t^\varphi)dt\text{a.e.}, [[/math]]
which implies that

[[math]] \lim_{n\to\infty}\int_{\R^d} \varphi(x)d\mu_n(x)=\int_0^K\mu_n(E_t^\varphi)dt\xrightarrow{n\to\infty} \int_0^K\mu(E_t^\varphi)dt=\int_{\R^d}\varphi(x)d\mu(x). [[/math]]

Consequences: We look at the case of [math]d=1[/math]. Let [math](X_n)_{n\geq 1}[/math] be a sequence of r.v.'s with values in [math]\R[/math] and let [math]X[/math] be a r.v. with values in [math]\R[/math]. One can show that

[[math]] \lim_{n\to\infty\atop law}X_n=X\Longleftrightarrow \lim_{n\to\infty}F_{X_n}(t)=F_X(t). [[/math]]

Proposition

Let [math](\mu_n)_{n\geq 1}[/math] and [math]\mu[/math] be probability measures on [math]\R^d[/math]. Let [math]H\subset C_b(\R^d)[/math] such that [math]\bar H\supset C_c(\R^d)[/math]. Then the following are equivalent.

  • [math]\lim_{n\to\infty\atop w}\mu_n=\mu.[/math]
  • For all [math]\varphi\in C_c(\R^d)[/math] we have
    [[math]] \lim_{n\to\infty}\int_{\R^d} \varphi d\mu_n=\int_{\R^d} \varphi d\mu. [[/math]]
  • For all [math]\varphi\in H[/math] we have
    [[math]] \lim_{n\to\infty}\int_{\R^d}\varphi d\mu_n=\int_{\R^d} \varphi d\mu. [[/math]]


Show Proof

It is obvious that [math](i)\Longrightarrow (ii)[/math] and [math](i)\Longrightarrow (iii)[/math]. Therefore we first show [math](ii)\Longrightarrow (i):[/math] Let therefore [math]\varphi\in C_b(\R^d)[/math] and let [math](f_k)_{k\geq 1}\in C_c(\R^d)[/math] with [math]0\leq f_k\leq 1[/math] and [math]f_k\uparrow 1[/math] as [math]k\to\infty[/math]. Then for all [math]k\geq 1[/math] we get that [math]\varphi f_k\in C_c(\R^d)[/math] and hence

[[math]] \lim_{n\to\infty}\int_{\R^d}\varphi f_kd\mu_n=\int_{\R^d} \varphi f_k d\mu. [[/math]]
Moreover, we have

[[math]] \left\vert \int_{\R^d} \varphi d\mu-\int_{\R^d}\varphi f_k d\mu\right\vert\leq \sup_{x\in\R^d}\vert\varphi(x)\vert\left(1-\int_{\R^d} f_kd\mu\right) [[/math]]
and also

[[math]] \left\vert \int_{\R^d} \varphi d\mu_n-\int_{\R^d} \varphi f_k d\mu_n\right\vert\leq \sup_{x\in\R^d}\vert \varphi(x)\vert \left(1-\int_{\R^d} f_k d\mu_n\right). [[/math]]

Hence, for all [math]k\geq 1[/math] we get

[[math]] \begin{align*} \limsup_n \left\vert\int_{\R^d}\varphi d\mu-\int_{\R^d} \varphi d\mu_n\right\vert&\leq \sup_{x\in \R^d}\vert \varphi(x)\vert \limsup_n\left[\left(1-\int_{\R^d} f_k d\mu_n\right)+\left(1-\int_{\R^d} f_k d\mu\right)\right]\\ &=2\sup_{x\in\R^d}\vert\varphi(x)\vert\left(1-\int f_kd\mu\right)\xrightarrow{k\to\infty}0. \end{align*} [[/math]]
Now we show [math](iii)\Longrightarrow (ii):[/math] Let therefore [math]\varphi\in C_c(\R^d)[/math]. Then there is a sequence [math](\varphi_k)_{k\geq 1}\subset H[/math] such that [math]\|\varphi-\varphi_k\|_\infty\leq \frac{1}{k}[/math] for all [math]k\geq 1[/math]. This implies that

[[math]] \begin{multline*} \limsup_n\left\vert\int \varphi d\mu_n-\int \varphi d\mu\right\vert\\\leq \limsup_n\left(\left\vert\int \varphi d\mu_n-\int\varphi_k d\mu_n\right\vert+\underbrace{\left\vert\int \varphi_k d\mu_n-\int\varphi_k d\mu\right\vert}_{\xrightarrow{n\to\infty}0} +\left\vert \int \varphi_k d\mu-\int\varphi d\mu\right\vert\right)\leq \frac{2}{k}. \end{multline*} [[/math]]
The claim follows now for [math]k\to\infty[/math].

Theorem (Lèvy)

Let [math](\Omega,\A,\p)[/math] be a probability space. Let [math](\mu_n)_{n\geq 1}[/math] be a sequence of probability measures on [math]\R^d[/math] associated to a sequence of real r.v.'s [math](X_n)_{n\geq 1}[/math]. Moreover, let [math]\hat{\mu}_n(\xi)=\int_{\R^d}e^{i\xi x}d\mu_n(x)[/math] and [math]\Phi_X(\xi)=\E[e^{i\xi x}][/math]. Then for all [math]\xi\in\R^d[/math] we get

[[math]] \lim_{n\to\infty\atop w}\mu_n=\mu \Longleftrightarrow\lim_{n\to\infty}\hat\mu_n(\xi)=\hat\mu(\xi). [[/math]]

Equivalently, for all [math]\xi\in\R^d[/math] we get

[[math]] \lim_{n\to\infty\atop law}X_n=X\Longleftrightarrow\lim_{n\to\infty}\Phi_{X_n}(\xi)=\Phi_X(\xi). [[/math]]


Show Proof

It is obvious that [math]\lim_{n\to\infty\atop w}\mu_n=\mu[/math] implies that [math]\lim_{n\to\infty}\hat{\mu}_n(\xi)=\hat{\mu}(\xi)[/math]. Therefore [math]e^{i\xi X}[/math] is continuous and bounded. For notation conventions we deal with the case [math]d=1[/math]. Let therefore [math]f\in C_c(\R^d)[/math]. For [math]\sigma \gt 0[/math] we also note [math]g_\sigma(x)=\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{x^2}{2\sigma^2}}[/math]. \begin{exer} Show that [math]g_\sigma *f\xrightarrow{\sigma\to 0}f[/math] uniformly on [math]\R[/math]. \end{exer} \begin{exer} Show that if [math]\nu[/math] is a probability measure, then

[[math]] \int_\R g_\sigma * fd\nu=\int_\R f(x)(g_\sigma *\nu)(x)dx=\int_\R f(x)\frac{1}{\sigma\sqrt{2\pi}}\int_\R e^{i\xi x}g_{\frac{1}{\sigma}}(\xi)\hat{\nu}(\xi)d\xi dx. [[/math]]
\end{exer} Since [math]\lim_{n\to\infty}\hat{\mu}_n(\xi)=\hat{\mu}(\xi)[/math], we get by the dominated convergence theorem that

[[math]] \int_\R e^{i\xi x}g_{\frac{1}{\sigma}}(\xi)\hat{\mu}_n(\xi)d\xi\xrightarrow{n\to\infty}\int_\R e^{i\xi x}g_{\frac{1}{\sigma}}(\xi)\hat{\mu}(\xi)d\xi. [[/math]]
These quantities are bounded by 1, and hence we can apply the dominated convergence theorem to obtain

[[math]] \int_\R g_\sigma * fd\mu_n\xrightarrow{n\to\infty} \int_\R g_\sigma * fd\mu. [[/math]]
Let now [math]H:=\{\varphi=g_\sigma *f\mid \sigma \gt 0,f\in C_c(\R^d)\}\subset C_b(\R^d)[/math]. Since [math]f\in C_c(\R^d)[/math] we get that [math]\|g_\sigma *f-f\|_{\infty}\xrightarrow{n\to\infty}0[/math] and thus [math]\bar H\supset C_c(\R^d)[/math]. The result now follows from the previous proposition.

Theorem (Lévy)

Let [math](\mu_n)_{n\geq 1}[/math] be a sequence of probability measures on [math]\R^d[/math] with characteristic functions [math](\Phi_n)_{n\geq 1}[/math]. If [math]\Phi_n[/math] converges pointwise to a function [math]\Phi[/math] which is continuous at 0, then

[[math]] \lim_{n\to\infty\atop w}\mu_n=\mu [[/math]]
for some probability measure [math]\mu[/math] on [math]\R^d[/math].


Show Proof

No proof here.

Example

Let [math](X_n)_{n\geq 1}[/math] be a sequence of poisson r.v.'s with parameter [math]\lambda[/math]. Moreover, consider the sequence [math]Z_n=\frac{X_n-1}{\sqrt{n}}[/math]. Then we have

[[math]] \begin{multline*} E\left[e^{iu Z_n}\right]=\E\left[e^{iu\left(\frac{X_n-u}{\sqrt{n}}\right)}\right]=e^{-i\frac{u}{\sqrt{n}}}\E\left[e^{iu\frac{X_n}{\sqrt{n}}}\right]=e^{-i\frac{u}{\sqrt{n}}}e^{\lambda\left(e^{i \frac{u}{\sqrt{n}}}-1\right)}\\=e^{-i\frac{u}{\sqrt{n}}}e^{\lambda\left(i\frac{u}{\sqrt{n}}-\frac{u^2}{2n}+O\left(\frac{1}{n}\right)\right)}\xrightarrow{n\to\infty}e^{-\frac{u^2}{2}}. \end{multline*} [[/math]]

Since [math]\E\left[e^{iu\mathcal{N}(0,1)} \right]=e^{-\frac{u^2}{2}}[/math], we deduce that [math]\lim_{n\to\infty\atop law}Z_n=\lim_{n\to\infty\atop law}\frac{X_n-u}{\sqrt{n}}=\mathcal{N}(0,1)[/math]. Before stating and proving the central limit theorem, we give two extra results on convergence in law.

Theorem

Let [math](\Omega,\A,\p)[/math] be a probability space. Let [math](X_n)_{n\geq 1}[/math] be a sequence of r.v.'s and [math]X[/math] a r.v. and assume that [math]\lim_{n\to\infty\atop law}X_n=X[/math] and that [math]X[/math] is a.s. equal to a constant [math]a[/math]. Then

[[math]] \lim_{n\to\infty\atop\p}X_n=X. [[/math]]


Show Proof

Let [math]f(x):=\vert x-a\vert\land 1[/math]. Then [math]f[/math] is a continuous and bounded map and therefore [math]\lim_{n\to\infty}\E[f(X_n)]=\E[f(X)]=0[/math], i.e. [math]\lim_{n\to\infty}\E[\vert X_n-a\vert \land 1]=0[/math] which implies that [math]\lim_{n\to\infty\atop \p}X_n=X[/math].

Theorem

Let [math](\Omega,\A,\p)[/math] be a probability space. Let [math](X_n)_{n\geq 1}[/math] be a sequence of r.v.'s and [math]X[/math] be r.v. in [math]\R^d[/math]. Assume that [math]X_n[/math] has density [math]f_n[/math] for all [math]n\geq 1[/math] and [math]X[/math] has density [math]f[/math]. Moreover, assume that [math]\lim_{n\to\infty}f_n(x)=f(x)[/math], a.e. Then

[[math]] \lim_{n\to\infty\atop law}X_n=X. [[/math]]


Show Proof

We need to show that [math]\E[h(X_n)]\xrightarrow{n\to\infty}\E[h(X)][/math], where [math]h:\R^d\to\R[/math] is a bounded and measurable map and

[[math]] \E[h(X_n)]=\int_{\R^d}h(x)f_n(x)dx, [[/math]]

[[math]] \E[h(X)]=\int_{\R^d}h(x)f(x)dx. [[/math]]
Let [math]h:\R^d\to\R[/math] be a bounded and measurable map. Moreover, let [math]\alpha=\sup_{x\in\R^d}\vert h(x)\vert.[/math] Set [math]h_1(x)=h(x)+\alpha\geq 0[/math] and [math]h_2(x)=\alpha-h(x)\geq 0[/math]. So it follows that [math]h_1f_n\geq 0[/math] and [math]h_2f_n\geq 0[/math]. Moreover, we also get that

[[math]] h_1f_n\xrightarrow{n\to\infty}h_1f\text{a.e.} [[/math]]

[[math]] h_2f_n\xrightarrow{n\to\infty}h_2f\text{a.e.} [[/math]]
With Fatou's lemma we get

[[math]] \E[h_1(X)]=\int_{\R^d}h_1(x)f(x)dx\leq \liminf_n\int_{\R^d}h_1(x)f_n(x)dx=\liminf_n\E[h_1(X_n)]. [[/math]]
Similarly we get [math]\E[h_2(X)]\leq \liminf_n\E[h_2(X_n)][/math]. Now substitute [math]h_1(x)=h(x)+\alpha[/math] and [math]h_2(x)=\alpha-h(x)[/math], where we use [math]\liminf_n(-a_n)=\liminf_n(a_n)[/math] to obtain

[[math]] \limsup_n\E[h(X_n)]\leq \E[h(X)]\leq \liminf_n\E[h(X_n)], [[/math]]
which implies that

[[math]] \liminf_n\E[h(X_n)]=\limsup_n\E[h(X_n)]=\E[h(X)]. [[/math]]

General references

Moshayedi, Nima (2020). "Lectures on Probability Theory". arXiv:2010.16280 [math.PR].