Probability X: Convergences

name: inter-slide
class: left, middle, inverse

---
name: layout-general
layout: true
class: left, middle

.remark-slide-number .progress-bar-container {
  position: absolute;
  bottom: 0;
  height: 4px;
  display: block;
  left: 0;
  right: 0;
}

.remark-slide-number .progress-bar {
  height: 100%;
  background-color: red;
}
</style>

<div>
<style type="text/css">.xaringan-extra-logo {
width: 110px;
height: 128px;
z-index: 0;
background-image: url(./img/Universite_Paris_logo_horizontal.jpg);
background-size: contain;
background-repeat: no-repeat;
position: absolute;
top:1em;right:1em;
}
</style>
<script>(function () {
  let tries = 0
  function addLogo () {
    if (typeof slideshow === 'undefined') {
      tries += 1
      if (tries < 10) {
        setTimeout(addLogo, 100)
      }
    } else {
      document.querySelectorAll('.remark-slide-content:not(.hide_logo)')
        .forEach(function (slide) {
          const logo = document.createElement('a')
          logo.classList = 'xaringan-extra-logo'
          logo.href = 'http://master.math.univ-paris-diderot.fr/annee/m1-mi/'
          slide.appendChild(logo)
        })
    }
  }
  document.addEventListener('DOMContentLoaded', addLogo)
})()</script>
</div>

---
template: inter-slide

# Convergences in distribution

### 2021-11-16

#### [Probabilités Master I MIDS](http://stephane-v-boucheron.fr/courses/probability/)

#### [Stéphane Boucheron](http://stephane-v-boucheron.fr)

---
template: inter-slide

## <svg aria-hidden="true" role="img" viewBox="0 0 576 512" style="height:1em;width:1.12em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M0 117.66v346.32c0 11.32 11.43 19.06 21.94 14.86L160 416V32L20.12 87.95A32.006 32.006 0 0 0 0 117.66zM192 416l192 64V96L192 32v384zM554.06 33.16L416 96v384l139.88-55.95A31.996 31.996 0 0 0 576 394.34V48.02c0-11.32-11.43-19.06-21.94-14.86z"/></svg>

### [Weak convergence, vague convergence]()
### [Convergence in distribution]()
### [Portemanteau Theorem]()
### [Lévy continuity theorem]()
### [Relations between convergences]()
### [Central limit theorem]()
### [Weak convergence and transforms]()

---
template: inter-slide

## Motivation

## <svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M248 8C111 8 0 119 0 256s111 248 248 248 248-111 248-248S385 8 248 8zm80 168c17.7 0 32 14.3 32 32s-14.3 32-32 32-32-14.3-32-32 14.3-32 32-32zm-160 0c17.7 0 32 14.3 32 32s-14.3 32-32 32-32-14.3-32-32 14.3-32 32-32zm80 256c-60.6 0-134.5-38.3-143.8-93.3-2-11.8 9.3-21.6 20.7-17.9C155.1 330.5 200 336 248 336s92.9-5.5 123.1-15.2c11.3-3.7 22.6 6.1 20.7 17.9-9.3 55-83.2 93.3-143.8 93.3z"/></svg>

---

.pull-left[

PMFs  (densities) of
- Binom(250,0.02) (left)
- Binom(2500,0.002) (middle)
- Poisson(5) (right)
]

.pull-right[

Graphical inspection of
probability mass functions suggests that

> As `$n$` grows, Binomial distributions with parameters `$(n, \lambda/n)$` look more and more alike Poisson distribution with parameter `$\lambda$`.

`$$\lim_{n \uparrow \infty}  \underbrace{(1 +\lambda(s-1)/n )^n}_{\text{PGF of} \operatorname{Binom}(n,\lambda/n)} = \underbrace{\exp(\lambda(s-1))}_{\text{PGF of} \operatorname{Poi}(\lambda)}$$`

]

---

### Convergence of Binomial PMFs towards Poisson PMFs

- PMFs belong to `$\ell_1(\mathbb{N})$`

- We can score the proximity between two probability distributions `$P, Q$` over `$\mathbb{N}$` by computing the `$\ell_1$` distance between the associated PMFs `$p$` and `$q$`

`$$\sum_{k \in \mathbb{N}} \big|p(k) - q(k)\big|$$`

- Up to a factor of `$2$`, this is the _Total Variation_ distance between `$P$` and `$Q$`

- For Binomials and Poisson distributions with respective parameters `$(n, \lambda/n)$` and `$\lambda$`, meaningful bounds can be derived at modest price

---

### Law of rare events

.left-column[

Distance between

- Binom `$(n, 5/n)$`

and

- Poisson `$(5)$`

w.r.t. `$n$`

]

.right-column[

<div id="htmlwidget-5049e6b7ed0a9754932d" style="width:504px;height:288px;" class="plotly html-widget"></div>
<script type="application/json" data-for="htmlwidget-5049e6b7ed0a9754932d">{"x":{"data":[{"x":[1,2,3,4,5,6,7],"y":[-0.463858253704861,-1.5986538704182,-2.60992542402594,-3.61103408422475,-4.61114476899808,-5.61115583565025,-6.61115694212506],"text":["ns: 1e+01<br />difs: 3.436701e-01","ns: 1e+02<br />difs: 2.519684e-02","ns: 1e+03<br />difs: 2.455130e-03","ns: 1e+04<br />difs: 2.448871e-04","ns: 1e+05<br />difs: 2.448247e-05","ns: 1e+06<br />difs: 2.448185e-06","ns: 1e+07<br />difs: 2.448178e-07"],"type":"scatter","mode":"markers","marker":{"autocolorscale":false,"color":"rgba(0,0,0,1)","opacity":1,"size":5.66929133858268,"symbol":"circle","line":{"width":1.88976377952756,"color":"rgba(0,0,0,1)"}},"hoveron":"points","showlegend":false,"xaxis":"x","yaxis":"y","hoverinfo":"text","frame":null},{"x":[1,1.07594936708861,1.15189873417722,1.22784810126582,1.30379746835443,1.37974683544304,1.45569620253165,1.53164556962025,1.60759493670886,1.68354430379747,1.75949367088608,1.83544303797468,1.91139240506329,1.9873417721519,2.06329113924051,2.13924050632911,2.21518987341772,2.29113924050633,2.36708860759494,2.44303797468354,2.51898734177215,2.59493670886076,2.67088607594937,2.74683544303797,2.82278481012658,2.89873417721519,2.9746835443038,3.05063291139241,3.12658227848101,3.20253164556962,3.27848101265823,3.35443037974684,3.43037974683544,3.50632911392405,3.58227848101266,3.65822784810127,3.73417721518987,3.81012658227848,3.88607594936709,3.9620253164557,4.0379746835443,4.11392405063291,4.18987341772152,4.26582278481013,4.34177215189873,4.41772151898734,4.49367088607595,4.56962025316456,4.64556962025316,4.72151898734177,4.79746835443038,4.87341772151899,4.94936708860759,5.0253164556962,5.10126582278481,5.17721518987342,5.25316455696203,5.32911392405063,5.40506329113924,5.48101265822785,5.55696202531646,5.63291139240506,5.70886075949367,5.78481012658228,5.86075949367089,5.93670886075949,6.0126582278481,6.08860759493671,6.16455696202532,6.24050632911392,6.31645569620253,6.39240506329114,6.46835443037975,6.54430379746835,6.62025316455696,6.69620253164557,6.77215189873418,6.84810126582278,6.92405063291139,7],"y":[-0.537977096232072,-0.615196226450961,-0.692415356669849,-0.769634486888737,-0.846853617107626,-0.924072747326514,-1.0012918775454,-1.07851100776429,-1.15573013798318,-1.23294926820207,-1.31016839842096,-1.38738752863984,-1.46460665885873,-1.54182578907762,-1.61904491929651,-1.6962640495154,-1.77348317973429,-1.85070230995317,-1.92792144017206,-2.00514057039095,-2.08235970060984,-2.15957883082873,-2.23679796104762,-2.3140170912665,-2.39123622148539,-2.46845535170428,-2.54567448192317,-2.62289361214206,-2.70011274236095,-2.77733187257984,-2.85455100279872,-2.93177013301761,-3.0089892632365,-3.08620839345539,-3.16342752367428,-3.24064665389316,-3.31786578411205,-3.39508491433094,-3.47230404454983,-3.54952317476872,-3.62674230498761,-3.7039614352065,-3.78118056542538,-3.85839969564427,-3.93561882586316,-4.01283795608205,-4.09005708630094,-4.16727621651982,-4.24449534673871,-4.3217144769576,-4.39893360717649,-4.47615273739538,-4.55337186761427,-4.63059099783316,-4.70781012805204,-4.78502925827093,-4.86224838848982,-4.93946751870871,-5.0166866489276,-5.09390577914648,-5.17112490936537,-5.24834403958426,-5.32556316980315,-5.40278230002204,-5.48000143024093,-5.55722056045982,-5.6344396906787,-5.71165882089759,-5.78887795111648,-5.86609708133537,-5.94331621155426,-6.02053534177315,-6.09775447199203,-6.17497360221092,-6.25219273242981,-6.3294118626487,-6.40663099286759,-6.48385012308648,-6.56106925330536,-6.63828838352425],"text":["ns: 1.000000<br />difs: -0.5379771","ns: 1.075949<br />difs: -0.6151962","ns: 1.151899<br />difs: -0.6924154","ns: 1.227848<br />difs: -0.7696345","ns: 1.303797<br />difs: -0.8468536","ns: 1.379747<br />difs: -0.9240727","ns: 1.455696<br />difs: -1.0012919","ns: 1.531646<br />difs: -1.0785110","ns: 1.607595<br />difs: -1.1557301","ns: 1.683544<br />difs: -1.2329493","ns: 1.759494<br />difs: -1.3101684","ns: 1.835443<br />difs: -1.3873875","ns: 1.911392<br />difs: -1.4646067","ns: 1.987342<br />difs: -1.5418258","ns: 2.063291<br />difs: -1.6190449","ns: 2.139241<br />difs: -1.6962640","ns: 2.215190<br />difs: -1.7734832","ns: 2.291139<br />difs: -1.8507023","ns: 2.367089<br />difs: -1.9279214","ns: 2.443038<br />difs: -2.0051406","ns: 2.518987<br />difs: -2.0823597","ns: 2.594937<br />difs: -2.1595788","ns: 2.670886<br />difs: -2.2367980","ns: 2.746835<br />difs: -2.3140171","ns: 2.822785<br />difs: -2.3912362","ns: 2.898734<br />difs: -2.4684554","ns: 2.974684<br />difs: -2.5456745","ns: 3.050633<br />difs: -2.6228936","ns: 3.126582<br />difs: -2.7001127","ns: 3.202532<br />difs: -2.7773319","ns: 3.278481<br />difs: -2.8545510","ns: 3.354430<br />difs: -2.9317701","ns: 3.430380<br />difs: -3.0089893","ns: 3.506329<br />difs: -3.0862084","ns: 3.582278<br />difs: -3.1634275","ns: 3.658228<br />difs: -3.2406467","ns: 3.734177<br />difs: -3.3178658","ns: 3.810127<br />difs: -3.3950849","ns: 3.886076<br />difs: -3.4723040","ns: 3.962025<br />difs: -3.5495232","ns: 4.037975<br />difs: -3.6267423","ns: 4.113924<br />difs: -3.7039614","ns: 4.189873<br />difs: -3.7811806","ns: 4.265823<br />difs: -3.8583997","ns: 4.341772<br />difs: -3.9356188","ns: 4.417722<br />difs: -4.0128380","ns: 4.493671<br />difs: -4.0900571","ns: 4.569620<br />difs: -4.1672762","ns: 4.645570<br />difs: -4.2444953","ns: 4.721519<br />difs: -4.3217145","ns: 4.797468<br />difs: -4.3989336","ns: 4.873418<br />difs: -4.4761527","ns: 4.949367<br />difs: -4.5533719","ns: 5.025316<br />difs: -4.6305910","ns: 5.101266<br />difs: -4.7078101","ns: 5.177215<br />difs: -4.7850293","ns: 5.253165<br />difs: -4.8622484","ns: 5.329114<br />difs: -4.9394675","ns: 5.405063<br />difs: -5.0166866","ns: 5.481013<br />difs: -5.0939058","ns: 5.556962<br />difs: -5.1711249","ns: 5.632911<br />difs: -5.2483440","ns: 5.708861<br />difs: -5.3255632","ns: 5.784810<br />difs: -5.4027823","ns: 5.860759<br />difs: -5.4800014","ns: 5.936709<br />difs: -5.5572206","ns: 6.012658<br />difs: -5.6344397","ns: 6.088608<br />difs: -5.7116588","ns: 6.164557<br />difs: -5.7888780","ns: 6.240506<br />difs: -5.8660971","ns: 6.316456<br />difs: -5.9433162","ns: 6.392405<br />difs: -6.0205353","ns: 6.468354<br />difs: -6.0977545","ns: 6.544304<br />difs: -6.1749736","ns: 6.620253<br />difs: -6.2521927","ns: 6.696203<br />difs: -6.3294119","ns: 6.772152<br />difs: -6.4066310","ns: 6.848101<br />difs: -6.4838501","ns: 6.924051<br />difs: -6.5610693","ns: 7.000000<br />difs: -6.6382884"],"type":"scatter","mode":"lines","name":"fitted values","line":{"width":1.88976377952756,"color":"rgba(51,102,255,1)","dash":"dash"},"hoveron":"points","showlegend":false,"xaxis":"x","yaxis":"y","hoverinfo":"text","frame":null},{"x":[1,1.07594936708861,1.15189873417722,1.22784810126582,1.30379746835443,1.37974683544304,1.45569620253165,1.53164556962025,1.60759493670886,1.68354430379747,1.75949367088608,1.83544303797468,1.91139240506329,1.9873417721519,2.06329113924051,2.13924050632911,2.21518987341772,2.29113924050633,2.36708860759494,2.44303797468354,2.51898734177215,2.59493670886076,2.67088607594937,2.74683544303797,2.82278481012658,2.89873417721519,2.9746835443038,3.05063291139241,3.12658227848101,3.20253164556962,3.27848101265823,3.35443037974684,3.43037974683544,3.50632911392405,3.58227848101266,3.65822784810127,3.73417721518987,3.81012658227848,3.88607594936709,3.9620253164557,4.0379746835443,4.11392405063291,4.18987341772152,4.26582278481013,4.34177215189873,4.41772151898734,4.49367088607595,4.56962025316456,4.64556962025316,4.72151898734177,4.79746835443038,4.87341772151899,4.94936708860759,5.0253164556962,5.10126582278481,5.17721518987342,5.25316455696203,5.32911392405063,5.40506329113924,5.48101265822785,5.55696202531646,5.63291139240506,5.70886075949367,5.78481012658228,5.86075949367089,5.93670886075949,6.0126582278481,6.08860759493671,6.16455696202532,6.24050632911392,6.31645569620253,6.39240506329114,6.46835443037975,6.54430379746835,6.62025316455696,6.69620253164557,6.77215189873418,6.84810126582278,6.92405063291139,7,7,7,6.92405063291139,6.84810126582278,6.77215189873418,6.69620253164557,6.62025316455696,6.54430379746835,6.46835443037975,6.39240506329114,6.31645569620253,6.24050632911392,6.16455696202532,6.08860759493671,6.0126582278481,5.93670886075949,5.86075949367089,5.78481012658228,5.70886075949367,5.63291139240506,5.55696202531646,5.48101265822785,5.40506329113924,5.32911392405063,5.25316455696203,5.17721518987342,5.10126582278481,5.0253164556962,4.94936708860759,4.87341772151899,4.79746835443038,4.72151898734177,4.64556962025316,4.56962025316456,4.49367088607595,4.41772151898734,4.34177215189873,4.26582278481013,4.18987341772152,4.11392405063291,4.0379746835443,3.9620253164557,3.88607594936709,3.81012658227848,3.73417721518987,3.65822784810127,3.58227848101266,3.50632911392405,3.43037974683544,3.35443037974684,3.27848101265823,3.20253164556962,3.12658227848101,3.05063291139241,2.9746835443038,2.89873417721519,2.82278481012658,2.74683544303797,2.67088607594937,2.59493670886076,2.51898734177215,2.44303797468354,2.36708860759494,2.29113924050633,2.21518987341772,2.13924050632911,2.06329113924051,1.9873417721519,1.91139240506329,1.83544303797468,1.75949367088608,1.68354430379747,1.60759493670886,1.53164556962025,1.45569620253165,1.37974683544304,1.30379746835443,1.22784810126582,1.15189873417722,1.07594936708861,1,1],"y":[-0.617543608006086,-0.693373721941582,-0.769215291473462,-0.845068944199993,-0.920935348277997,-0.996815215087142,-1.07270930201565,-1.14861841535702,-1.22454341330222,-1.3004852090058,-1.37644477369683,-1.45242313979652,-1.52842140399407,-1.60444073021953,-1.68048235243847,-1.75654757717671,-1.83263778566537,-1.90875443547657,-1.98489906149883,-2.06107327607985,-2.13727876814249,-2.21351730106086,-2.28979070906714,-2.36610089195049,-2.44244980780846,-2.51883946362264,-2.5952719034578,-2.67174919412936,-2.74827340825235,-2.82484660467673,-2.90147080642925,-2.97814797641903,-3.05487999131698,-3.13166861417964,-3.20851546654363,-3.28542200085368,-3.36238947418856,-3.43941892430044,-3.51651114897065,-3.59366668960297,-3.67088581982186,-3.74816853962731,-3.82551457539488,-3.90292338572078,-3.98039417282367,-4.0579258989514,-4.13551730702519,-4.2131669446003,-4.29087319014013,-4.36863428058813,-4.44644833927338,-4.52431340328678,-4.60222744960157,-4.68018841936779,-4.7581942399704,-4.83624284459399,-4.91433218917381,-4.99246026672823,-5.07062511915973,-5.14882484667914,-5.22705761505427,-5.30532166091103,-5.38361529532654,-5.46193690595313,-5.54028495790224,-5.61865799360178,-5.69705463182061,-5.77547356603293,-5.85391356227316,-5.93237345661124,-6.01085215235799,-6.08934861709218,-6.16786187958476,-6.24639102668117,-6.32493520019044,-6.40349359381907,-6.48206545017884,-6.56065005789009,-6.63924674879599,-6.71785489529827,-6.71785489529827,-6.55872187175024,-6.48289175781474,-6.40705018828286,-6.33119653555633,-6.25533013147833,-6.17945026466918,-6.10355617774067,-6.0276470643993,-5.95172206645411,-5.87578027075052,-5.79982070605949,-5.7238423399598,-5.64784407576225,-5.57182474953679,-5.49578312731786,-5.41971790257962,-5.34362769409095,-5.26751104427976,-5.19136641825749,-5.11519220367648,-5.03898671161383,-4.96274817869546,-4.88647477068919,-4.81016458780583,-4.73381567194787,-4.65742601613369,-4.58099357629852,-4.50451628562696,-4.42799207150398,-4.3514188750796,-4.27479467332707,-4.1981175033373,-4.12138548843935,-4.04459686557668,-3.96775001321269,-3.89084347890265,-3.81387600556776,-3.73684655545589,-3.65975433078568,-3.58259879015336,-3.50537965993447,-3.42809694012901,-3.35075090436144,-3.27334209403554,-3.19587130693265,-3.11833958080492,-3.04074817273114,-2.96309853515602,-2.8853922896162,-2.8076311991682,-2.72981714048294,-2.65195207646955,-2.57403803015476,-2.49607706038854,-2.41807123978593,-2.34002263516233,-2.26193329058251,-2.18380521302809,-2.10564036059659,-2.02744063307719,-1.94920786470205,-1.87094381884529,-1.79265018442978,-1.7143285738032,-1.63598052185409,-1.55760748615455,-1.47921084793571,-1.40079191372339,-1.32235191748317,-1.24389202314508,-1.16541332739833,-1.08691686266414,-1.00840360017156,-0.929874453075151,-0.851330279565886,-0.772771885937255,-0.694200029577482,-0.615615421866236,-0.537018730960339,-0.458410584458059,-0.617543608006086],"text":["ns: 1.000000<br />difs: -0.5379771","ns: 1.075949<br />difs: -0.6151962","ns: 1.151899<br />difs: -0.6924154","ns: 1.227848<br />difs: -0.7696345","ns: 1.303797<br />difs: -0.8468536","ns: 1.379747<br />difs: -0.9240727","ns: 1.455696<br />difs: -1.0012919","ns: 1.531646<br />difs: -1.0785110","ns: 1.607595<br />difs: -1.1557301","ns: 1.683544<br />difs: -1.2329493","ns: 1.759494<br />difs: -1.3101684","ns: 1.835443<br />difs: -1.3873875","ns: 1.911392<br />difs: -1.4646067","ns: 1.987342<br />difs: -1.5418258","ns: 2.063291<br />difs: -1.6190449","ns: 2.139241<br />difs: -1.6962640","ns: 2.215190<br />difs: -1.7734832","ns: 2.291139<br />difs: -1.8507023","ns: 2.367089<br />difs: -1.9279214","ns: 2.443038<br />difs: -2.0051406","ns: 2.518987<br />difs: -2.0823597","ns: 2.594937<br />difs: -2.1595788","ns: 2.670886<br />difs: -2.2367980","ns: 2.746835<br />difs: -2.3140171","ns: 2.822785<br />difs: -2.3912362","ns: 2.898734<br />difs: -2.4684554","ns: 2.974684<br />difs: -2.5456745","ns: 3.050633<br />difs: -2.6228936","ns: 3.126582<br />difs: -2.7001127","ns: 3.202532<br />difs: -2.7773319","ns: 3.278481<br />difs: -2.8545510","ns: 3.354430<br />difs: -2.9317701","ns: 3.430380<br />difs: -3.0089893","ns: 3.506329<br />difs: -3.0862084","ns: 3.582278<br />difs: -3.1634275","ns: 3.658228<br />difs: -3.2406467","ns: 3.734177<br />difs: -3.3178658","ns: 3.810127<br />difs: -3.3950849","ns: 3.886076<br />difs: -3.4723040","ns: 3.962025<br />difs: -3.5495232","ns: 4.037975<br />difs: -3.6267423","ns: 4.113924<br />difs: -3.7039614","ns: 4.189873<br />difs: -3.7811806","ns: 4.265823<br />difs: -3.8583997","ns: 4.341772<br />difs: -3.9356188","ns: 4.417722<br />difs: -4.0128380","ns: 4.493671<br />difs: -4.0900571","ns: 4.569620<br />difs: -4.1672762","ns: 4.645570<br />difs: -4.2444953","ns: 4.721519<br />difs: -4.3217145","ns: 4.797468<br />difs: -4.3989336","ns: 4.873418<br />difs: -4.4761527","ns: 4.949367<br />difs: -4.5533719","ns: 5.025316<br />difs: -4.6305910","ns: 5.101266<br />difs: -4.7078101","ns: 5.177215<br />difs: -4.7850293","ns: 5.253165<br />difs: -4.8622484","ns: 5.329114<br />difs: -4.9394675","ns: 5.405063<br />difs: -5.0166866","ns: 5.481013<br />difs: -5.0939058","ns: 5.556962<br />difs: -5.1711249","ns: 5.632911<br />difs: -5.2483440","ns: 5.708861<br />difs: -5.3255632","ns: 5.784810<br />difs: -5.4027823","ns: 5.860759<br />difs: -5.4800014","ns: 5.936709<br />difs: -5.5572206","ns: 6.012658<br />difs: -5.6344397","ns: 6.088608<br />difs: -5.7116588","ns: 6.164557<br />difs: -5.7888780","ns: 6.240506<br />difs: -5.8660971","ns: 6.316456<br />difs: -5.9433162","ns: 6.392405<br />difs: -6.0205353","ns: 6.468354<br />difs: -6.0977545","ns: 6.544304<br />difs: -6.1749736","ns: 6.620253<br />difs: -6.2521927","ns: 6.696203<br />difs: -6.3294119","ns: 6.772152<br />difs: -6.4066310","ns: 6.848101<br />difs: -6.4838501","ns: 6.924051<br />difs: -6.5610693","ns: 7.000000<br />difs: -6.6382884","ns: 7.000000<br />difs: -6.6382884","ns: 7.000000<br />difs: -6.6382884","ns: 6.924051<br />difs: -6.5610693","ns: 6.848101<br />difs: -6.4838501","ns: 6.772152<br />difs: -6.4066310","ns: 6.696203<br />difs: -6.3294119","ns: 6.620253<br />difs: -6.2521927","ns: 6.544304<br />difs: -6.1749736","ns: 6.468354<br />difs: -6.0977545","ns: 6.392405<br />difs: -6.0205353","ns: 6.316456<br />difs: -5.9433162","ns: 6.240506<br />difs: -5.8660971","ns: 6.164557<br />difs: -5.7888780","ns: 6.088608<br />difs: -5.7116588","ns: 6.012658<br />difs: -5.6344397","ns: 5.936709<br />difs: -5.5572206","ns: 5.860759<br />difs: -5.4800014","ns: 5.784810<br />difs: -5.4027823","ns: 5.708861<br />difs: -5.3255632","ns: 5.632911<br />difs: -5.2483440","ns: 5.556962<br />difs: -5.1711249","ns: 5.481013<br />difs: -5.0939058","ns: 5.405063<br />difs: -5.0166866","ns: 5.329114<br />difs: -4.9394675","ns: 5.253165<br />difs: -4.8622484","ns: 5.177215<br />difs: -4.7850293","ns: 5.101266<br />difs: -4.7078101","ns: 5.025316<br />difs: -4.6305910","ns: 4.949367<br />difs: -4.5533719","ns: 4.873418<br />difs: -4.4761527","ns: 4.797468<br />difs: -4.3989336","ns: 4.721519<br />difs: -4.3217145","ns: 4.645570<br />difs: -4.2444953","ns: 4.569620<br />difs: -4.1672762","ns: 4.493671<br />difs: -4.0900571","ns: 4.417722<br />difs: -4.0128380","ns: 4.341772<br />difs: -3.9356188","ns: 4.265823<br />difs: -3.8583997","ns: 4.189873<br />difs: -3.7811806","ns: 4.113924<br />difs: -3.7039614","ns: 4.037975<br />difs: -3.6267423","ns: 3.962025<br />difs: -3.5495232","ns: 3.886076<br />difs: -3.4723040","ns: 3.810127<br />difs: -3.3950849","ns: 3.734177<br />difs: -3.3178658","ns: 3.658228<br />difs: -3.2406467","ns: 3.582278<br />difs: -3.1634275","ns: 3.506329<br />difs: -3.0862084","ns: 3.430380<br />difs: -3.0089893","ns: 3.354430<br />difs: -2.9317701","ns: 3.278481<br />difs: -2.8545510","ns: 3.202532<br />difs: -2.7773319","ns: 3.126582<br />difs: -2.7001127","ns: 3.050633<br />difs: -2.6228936","ns: 2.974684<br />difs: -2.5456745","ns: 2.898734<br />difs: -2.4684554","ns: 2.822785<br />difs: -2.3912362","ns: 2.746835<br />difs: -2.3140171","ns: 2.670886<br />difs: -2.2367980","ns: 2.594937<br />difs: -2.1595788","ns: 2.518987<br />difs: -2.0823597","ns: 2.443038<br />difs: -2.0051406","ns: 2.367089<br />difs: -1.9279214","ns: 2.291139<br />difs: -1.8507023","ns: 2.215190<br />difs: -1.7734832","ns: 2.139241<br />difs: -1.6962640","ns: 2.063291<br />difs: -1.6190449","ns: 1.987342<br />difs: -1.5418258","ns: 1.911392<br />difs: -1.4646067","ns: 1.835443<br />difs: -1.3873875","ns: 1.759494<br />difs: -1.3101684","ns: 1.683544<br />difs: -1.2329493","ns: 1.607595<br />difs: -1.1557301","ns: 1.531646<br />difs: -1.0785110","ns: 1.455696<br />difs: -1.0012919","ns: 1.379747<br />difs: -0.9240727","ns: 1.303797<br />difs: -0.8468536","ns: 1.227848<br />difs: -0.7696345","ns: 1.151899<br />difs: -0.6924154","ns: 1.075949<br />difs: -0.6151962","ns: 1.000000<br />difs: -0.5379771","ns: 1.000000<br />difs: -0.5379771"],"type":"scatter","mode":"lines","line":{"width":1.88976377952756,"color":"transparent","dash":"dash"},"fill":"toself","fillcolor":"rgba(153,153,153,0.4)","hoveron":"points","hoverinfo":"x+y","showlegend":false,"xaxis":"x","yaxis":"y","frame":null}],"layout":{"margin":{"t":28.7853881278539,"r":7.30593607305936,"b":42.7397260273973,"l":54.7945205479452},"plot_bgcolor":"rgba(255,255,255,1)","paper_bgcolor":"rgba(255,255,255,1)","font":{"color":"rgba(0,0,0,1)","family":"","size":14.6118721461187},"xaxis":{"domain":[0,1],"automargin":true,"type":"linear","autorange":false,"range":[0.7,7.3],"tickmode":"array","ticktext":["1e+02","1e+04","1e+06"],"tickvals":[2,4,6],"categoryorder":"array","categoryarray":["1e+02","1e+04","1e+06"],"nticks":null,"ticks":"outside","tickcolor":"rgba(51,51,51,1)","ticklen":3.65296803652968,"tickwidth":0.66417600664176,"showticklabels":true,"tickfont":{"color":"rgba(77,77,77,1)","family":"","size":11.689497716895},"tickangle":-0,"showline":false,"linecolor":null,"linewidth":0,"showgrid":true,"gridcolor":"rgba(235,235,235,1)","gridwidth":0.66417600664176,"zeroline":false,"anchor":"y","title":{"text":"n","font":{"color":"rgba(0,0,0,1)","family":"","size":14.6118721461187}},"hoverformat":".2f"},"yaxis":{"domain":[0,1],"automargin":true,"type":"linear","autorange":false,"range":[-7.03082711084028,-0.145438368916048],"tickmode":"array","ticktext":["1e-06","1e-04","1e-02"],"tickvals":[-6,-4,-2],"categoryorder":"array","categoryarray":["1e-06","1e-04","1e-02"],"nticks":null,"ticks":"outside","tickcolor":"rgba(51,51,51,1)","ticklen":3.65296803652968,"tickwidth":0.66417600664176,"showticklabels":true,"tickfont":{"color":"rgba(77,77,77,1)","family":"","size":11.689497716895},"tickangle":-0,"showline":false,"linecolor":null,"linewidth":0,"showgrid":true,"gridcolor":"rgba(235,235,235,1)","gridwidth":0.66417600664176,"zeroline":false,"anchor":"x","title":{"text":"Total variation distance","font":{"color":"rgba(0,0,0,1)","family":"","size":14.6118721461187}},"hoverformat":".2f"},"shapes":[{"type":"rect","fillcolor":"transparent","line":{"color":"rgba(51,51,51,1)","width":0.66417600664176,"linetype":"solid"},"yref":"paper","xref":"paper","x0":0,"x1":1,"y0":0,"y1":1}],"showlegend":false,"legend":{"bgcolor":"rgba(255,255,255,1)","bordercolor":"transparent","borderwidth":1.88976377952756,"font":{"color":"rgba(0,0,0,1)","family":"","size":11.689497716895}},"hovermode":"closest","barmode":"relative"},"config":{"doubleClick":"reset","showSendToCloud":false},"source":"A","attrs":{"51e785bf528":{"x":{},"y":{},"type":"scatter"},"51e5d3c52d3":{"x":{},"y":{}}},"cur_data":"51e785bf528","visdat":{"51e785bf528":["function (y) ","x"],"51e5d3c52d3":["function (y) ","x"]},"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.2,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>

]

Logarithmic scales on both axes <svg aria-hidden="true" role="img" viewBox="0 0 576 512" style="height:1em;width:1.12em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M569.517 440.013C587.975 472.007 564.806 512 527.94 512H48.054c-36.937 0-59.999-40.055-41.577-71.987L246.423 23.985c18.467-32.009 64.72-31.951 83.154 0l239.94 416.028zM288 354c-25.405 0-46 20.595-46 46s20.595 46 46 46 46-20.595 46-46-20.595-46-46-46zm-43.673-165.346l7.418 136c.347 6.364 5.609 11.346 11.982 11.346h48.546c6.373 0 11.635-4.982 11.982-11.346l7.418-136c.375-6.874-5.098-12.654-11.982-12.654h-63.383c-6.884 0-12.356 5.78-11.981 12.654z"/></svg>

---

## <svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M328.2 255.8h151.6c9.1 0 16.8-7.7 16.2-16.8-5.1-75.8-44.4-142.2-102.5-184.2-7.4-5.3-17.9-2.9-22.7 4.8L290.4 188c22.6 14.3 37.8 39.2 37.8 67.8zm-37.8 67.7c-12.3 7.7-26.8 12.4-42.4 12.4-15.6 0-30-4.7-42.4-12.4L125.2 452c-4.8 7.7-2.4 18.1 5.6 22.4C165.7 493.2 205.6 504 248 504s82.3-10.8 117.2-29.6c8-4.3 10.4-14.8 5.6-22.4l-80.4-128.5zM248 303.8c26.5 0 48-21.5 48-48s-21.5-48-48-48-48 21.5-48 48 21.5 48 48 48zm-231.8-48h151.6c0-28.6 15.2-53.5 37.8-67.7L125.2 59.7c-4.8-7.7-15.3-10.2-22.7-4.8C44.4 96.9 5.1 163.3 0 239.1c-.6 9 7.1 16.7 16.2 16.7z"/></svg>

We are talking about probability distributions, not random variables

We are not talking about Binomial and Poisson random variables leaving on the same
probability space but on probability distributions of random variables leaving
on possibly different probability spaces

The set of probability distributions
over some measurable space `$(\Omega, \mathcal{F})$`
can be equipped with a variety of topologies

We shall focus on the topology defined
by _convergence in distribution_ also called _weak convergence_.

---

### Roadmap

.pull-left[
We introduce _weak and vague convergences_ for sequences of probability distributions

Weak convergence induces the definition of _convergence in distribution_ for random variables that possibly live on different probability spaces

The Portemanteau Theorem lists a number of alternative and equivalent characterizations of convergence in distribution.

Alternative characterizations are useful in two respects: they may be easier to check; they may supply a larger range of applications

]

.pull-right[

We state and prove the Lévy continuity theorem

The Lévy continuity theorem relates convergence in distribution with pointwise convergence of characteristic functions

The Lévy continuity theorem could be one more line in the statement
of the Portemanteau Theorem

But it stands out
because it provides us with a concise proof of the Central Limit Theorem
for normalized sums of centered i.i.d. random variables

]

---
name: weakandvague
class: middle, center, inverse

## Weak convergence, vague convergence

---

Weak convergence of probability measures assesses the proximity of probability measures by
comparing their action on a _collection of test functions_

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Definition: Weak convergence

A sequence of probability distributions `$(P_n)_{n \in \mathbb{N}}$`
over `$\mathbb{R}^k$` converges _weakly_ towards probability distribution   `$P$` (on `$\mathbb{R}^k$`)

iff

for any bounded and continuous function `$f: \mathbb{R}^k \to \mathbb{R}$`,

`$$\lim_n \mathbb{E}_{P_n} [f] =  \mathbb{E}_P [f]$$`

]

---

- <svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M248 8C111 8 0 119 0 256s111 248 248 248 248-111 248-248S385 8 248 8zm80 168c17.7 0 32 14.3 32 32s-14.3 32-32 32-32-14.3-32-32 14.3-32 32-32zm-160 0c17.7 0 32 14.3 32 32s-14.3 32-32 32-32-14.3-32-32 14.3-32 32-32zm194.8 170.2C334.3 380.4 292.5 400 248 400s-86.3-19.6-114.8-53.8c-13.6-16.3 11-36.7 24.6-20.5 22.4 26.9 55.2 42.2 90.2 42.2s67.8-15.4 90.2-42.2c13.4-16.2 38.1 4.2 24.6 20.5z"/></svg> The there is some flexibility in the choice of the class of test functions

- <svg aria-hidden="true" role="img" viewBox="0 0 576 512" style="height:1em;width:1.12em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M569.517 440.013C587.975 472.007 564.806 512 527.94 512H48.054c-36.937 0-59.999-40.055-41.577-71.987L246.423 23.985c18.467-32.009 64.72-31.951 83.154 0l239.94 416.028zM288 354c-25.405 0-46 20.595-46 46s20.595 46 46 46 46-20.595 46-46-20.595-46-46-46zm-43.673-165.346l7.418 136c.347 6.364 5.609 11.346 11.982 11.346h48.546c6.373 0 11.635-4.982 11.982-11.346l7.418-136c.375-6.874-5.098-12.654-11.982-12.654h-63.383c-6.884 0-12.356 5.78-11.981 12.654z"/></svg> This choice is not unlimited

> If we restrict the collection of test functions to continuous functions with _compact support_ (which are always bounded), we obtain a _different_ notion of convergence.

---
name: vagueconv

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Definition  Vague convergence

A sequence of probability distributions `$(P_n)_{n\in\mathbb{N}}$` over `$\mathbb{R}^k$` converges _vaguely_ towards measure `$\mu$` (on `$\mathbb{R}^k$`)

iff

for any  continuous function `$f$` with _compact support_ from `$\mathbb{R}^k \to \mathbb{R}$`,

`$$\lim_n \mathbb{E}_{P_n} [f] = \int f \mathrm{d}\mu$$`

]

---

### Example

Consider the sequence of probability masses over the integers `$(\delta_n)_{n \in \mathbb{N}}$`.

- This sequence converges vaguely towards the null measure

- This sequence  does not converge weakly

---
class: middle, center, inverse
name: secConvDistribution

## Convergence in distribution

---

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Definition

A sequence  `$(X_n)_{n \in \mathbb{N}}$` of `$\mathbb{R}^k$`-valued random variables defined
on a sequence of probability spaces  `$(\Omega_n, \mathcal{F}_n, P_n)$` converges _in distribution_
towards `$X \sim \mathcal{L}$`

`$(P_n \circ X_n^{-1})_{n \in \mathbb{N}}$`  is weakly convergent towards  probability distribution  `$\mathcal{L}$` over `$(\mathbb{R}^k, \mathcal{B}(\mathbb{R}^k))$`

`$$X_n \rightsquigarrow X \qquad \text{or} \qquad  X_n \rightsquigarrow \mathcal{L}$$`

]

In order to check or use convergence in distribution, many equivalent characterizations are available.

Some of them are listed in the Portemanteau Theorem.

---
class: middle, center, inverse
name: portemanteau

## Portemanteau Theorem

---

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Portemanteau Theorem <svg aria-hidden="true" role="img" viewBox="0 0 640 512" style="height:1em;width:1.25em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M224 320h32V96h-32c-17.67 0-32 14.33-32 32v160c0 17.67 14.33 32 32 32zm352-32V128c0-17.67-14.33-32-32-32h-32v224h32c17.67 0 32-14.33 32-32zm48 96H128V16c0-8.84-7.16-16-16-16H16C7.16 0 0 7.16 0 16v32c0 8.84 7.16 16 16 16h48v368c0 8.84 7.16 16 16 16h82.94c-1.79 5.03-2.94 10.36-2.94 16 0 26.51 21.49 48 48 48s48-21.49 48-48c0-5.64-1.15-10.97-2.94-16h197.88c-1.79 5.03-2.94 10.36-2.94 16 0 26.51 21.49 48 48 48s48-21.49 48-48c0-5.64-1.15-10.97-2.94-16H624c8.84 0 16-7.16 16-16v-32c0-8.84-7.16-16-16-16zM480 96V48c0-26.51-21.49-48-48-48h-96c-26.51 0-48 21.49-48 48v272h192V96zm-48 0h-96V48h96v48z"/></svg>

A sequence of probability distributions `$(P_n)_{n \in \mathbb{N}}$` on `$\mathbb{R}^k$` converges weakly
towards a probability distribution `$P$` (on `$\mathbb{R}^k$`) iff one of the equivalent properties hold:

1. For every bounded continuous function `$f$` from `$\mathbb{R}^k$` to
`$\mathbb{R}$`, the sequence `$\mathbb{E}_{P_n} [f]$` converges towards
`$\mathbb{E}_P [f]$`.
1. For every bounded uniformly continuous function `$f$` from
`$\mathbb{R}^k$` to `$\mathbb{R}$`, the sequence `$\mathbb{E}_{P_n} [f]$`
converges towards `$\mathbb{E}_P [f]$`.
1. For every bounded Lipschitz function  `$f$` from `$\mathbb{R}^k$` to `$\mathbb{R}$`, the sequence `$\mathbb{E}_{P_n} [f]$` converges towards `$\mathbb{E}_P [f]$`.
1. For every `$P$`-almost surely bounded  and  continuous function `$f$` from `$\mathbb{R}^k$` to `$\mathbb{R}$`, the sequence
`$(\mathbb{E}_{P_n} [f])$` converges towards `$\mathbb{E}_P [f]$`.
1. For every closed subset  `$F$` of `$\mathbb{R}^k$`, `$\limsup P_n (F) \leq  P(F).$`
1. For every open subset  `$O$` of `$\mathbb{R}^k$`, `$\liminf P_n (O) \geq  P(O).$`
1. For every `$A \in \mathcal{B}(\mathbb{R})$` such that `$P(A^\circ) = P(\overline{A})$` (the boundary of  `$A$` is `$P$`-negligible), `$\lim_n P_n(A)=P(A)$`.

]

---

### Proof <svg aria-hidden="true" role="img" viewBox="0 0 640 512" style="height:1em;width:1.25em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M224 320h32V96h-32c-17.67 0-32 14.33-32 32v160c0 17.67 14.33 32 32 32zm352-32V128c0-17.67-14.33-32-32-32h-32v224h32c17.67 0 32-14.33 32-32zm48 96H128V16c0-8.84-7.16-16-16-16H16C7.16 0 0 7.16 0 16v32c0 8.84 7.16 16 16 16h48v368c0 8.84 7.16 16 16 16h82.94c-1.79 5.03-2.94 10.36-2.94 16 0 26.51 21.49 48 48 48s48-21.49 48-48c0-5.64-1.15-10.97-2.94-16h197.88c-1.79 5.03-2.94 10.36-2.94 16 0 26.51 21.49 48 48 48s48-21.49 48-48c0-5.64-1.15-10.97-2.94-16H624c8.84 0 16-7.16 16-16v-32c0-8.84-7.16-16-16-16zM480 96V48c0-26.51-21.49-48-48-48h-96c-26.51 0-48 21.49-48 48v272h192V96zm-48 0h-96V48h96v48z"/></svg>

Implications `$1) \Rightarrow 2) \Rightarrow 3)$` are obvious.

Lévy's continuity theorem entails that `$3) \Rightarrow 1)$`.

`$4) \Rightarrow 1)$` is obvious.

That `$5) \Leftrightarrow 6)$` follows from the fact that the complement of a closed set
is an open set

---

`$5)$` and `$6)$` imply `$7)$`:

`$$\limsup_n P_n(\overline{A}) \leq P(\overline{A}) = P(A^\circ) \leq \liminf_n P_n(A^\circ)$$`

By monotony

`$$\liminf_n P_n(A^\circ) \leq \liminf_n P_n (A) \leq \limsup_n P_n(A) \leq \limsup_n P_n (\overline{A})$$`

Combining leads to

`$$\lim_n P_n(A) = \liminf_n P_n (A) = \limsup_n P_n(A) = P(A^\circ) = P(\overline{A})$$`

---

Let us check that `$3) \Rightarrow 5)$`. Let `$F$` be a closed subset of `$\mathbb{R}^k$`.

For `$x\in \mathbb{R}^k$`, let `$\mathrm{d}(x,F)$` denote the distance from `$x$` to `$F$`.

For `$m \in \mathbb{N}$`, let `$f_m(x) = \big(1 - m \mathrm{d}(x, F)\big)_+$`.

The function `$f_m$`
`$m$`-is Lifchitz, lower bounded by `$\mathbb{I}_F$`, and for every `$x \in \mathbb{R}^k$` `$\lim_m \downarrow f_m(x)= \mathbb{I}_F(x)$`

Weak convergence of `$P_n$` to `$P$` implies

`$$\lim_n \mathbb{E}_{P_n} f_m = \mathbb{E}_P f_m$$`

---

Hence for every `$m \in \mathbb{N}$`

`$$\limsup_n P_n(F) = \limsup_n \mathbb{E}_{P_n} \mathbb{I}_F \leq  \lim_n \mathbb{E}_{P_n} f_m  = \mathbb{E}_P f_m$$`

Taking the limit on the right side leads to

`$$\limsup_n P_n(F) = \limsup_n \mathbb{E}_{P_n} \mathbb{I}_F \leq \lim_m \downarrow \mathbb{E}_P f_m = \mathbb{E}_P \mathbb{I}_F =P (F)$$`

(teaking the Monotone Converence Theorem)

---

Checking `$7) \Rightarrow 1)$`

Let `$f$` be a bounded continuous function. Assume w.l.o.g. that `$f$` is non-negative and upper-bounded by `$1$`. Recall that for each `$\sigma$`-finite measure `$\mu$`

`$$\int f \mathrm{d}\mu = \int_{[0,\infty)} \mu\{f > t\} \mathrm{d}t$$`

This holds for all `$P_n$` and `$P$`. Hence

`$$\mathbb{E}_{P_n} f = \int_{[0,\infty)} P_n \{ f > t \} \mathrm{d}t$$`

As `$\overline{\{ f > t\}} = \{f \geq t\}$`

`$$\overline{\{ f > t\}} \setminus \{ f > t\}^\circ = \{f = t\}$$`

---

The  set of values `$t$` such that `$P \{ f = t\}>0$` is at most countable and thus Lebesgue-negligible

Let `$E$` be its complement. For `$t\in E$`, `$\lim_n P_n\{ f> t\} = P\{f >t\}$`.

`$$\begin{array}{rl}  \lim_n \mathbb{E}_{P_n} f    & = \lim_n \int_{[0, 1]} P_n \{ f >t \} \mathrm{d}t \\    & = \lim_n \int_{[0, 1]} P_n \{ f >t \} \mathbb{I}_E(t) \mathrm{d}t \\    & = \int_{[0, 1]} \lim_n  P_n \{ f >t \} \mathbb{I}_E(t) \mathrm{d}t \\    & = \int_{[0,1]} P\{f >t\}  \mathbb{I}_E(t) \mathrm{d}t \\    & = \int_{[0,1]} P\{f >t\}   \mathrm{d}t \\    & = \mathbb{E}_P f\end{array}$$`

---

### Illustration

- `$X_n \sim P_n = \delta_{1/n}$`  and `$X \sim \delta_0$`

- `$X_n ⇝ X$`

- Closed set `$F = (-\infty, 0]$`,
`$$P_n(F) = 0, \quad \forall n$$`
but `$P(F) = 1 \geq \limsup_n P_n(F)=0$`

- Open set `$O = ]0, \infty)$`,
`$$P_n(O) = 1, \quad \forall n$$`
but `$P(O) = 0 \leq \liminf_n P_n(O)=1$`

---

> For probability measures over `$(\mathbb{R}, \mathcal{B}(\mathbb{R}))$`, weak convergence is determined by cumulative distribution functions. This is sometimes taken as a definition of weak convergence in elementary books.

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Corollary

A sequence of probability measures   defined by their cumulative distribution functions `$(F_n)_n$` converges weakly towards a probability measure defined by cumulative distribution function `$F$`

iff

`$$\lim_n F_n(x) = F(x)$$`

at every `$x$` which is a  continuity point of `$F$`

]

Consequence of 7) in the statement of the Portemanteau Theorem: if `$x$` is a continuity point of `$F$`, then

`$$F(x) = P(-\infty, x] = P(-\infty, x[ = P(-\infty, x]^{\circ} = \lim_{x_n \uparrow x} F(x_n)$$`

---
name: weakconvquantiles

For probability measures over `$(\mathbb{R}, \mathcal{B}(\mathbb{R}))$`, weak convergence is also determined by quantile functions

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Theorem

A sequence of probability measures   defined by their quantile functions `$(F^\leftarrow_n)_n$` converges weakly towards a probability measure defined by quantile function `$F^\leftarrow$`

iff

`$$\lim_n F_n^\leftarrow(x) = F^\leftarrow(x)$$`

at every `$x$` which is a  continuity point of `$F^\leftarrow$`

]

---
name: asrepweakconv

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Proposition   Almost sure representation

If `$(X_n)_n ⇝  X$`,

then

there exists  `$(\Omega, \mathcal{F}, P)$` with random variables `$(Y_n)_n$`
and `$Y$`,   such that:

-  `$\forall n, X_n \sim Y_n$`,
-  `$X \sim Y$`,
-  and

`$$Y_n \to Y \qquad P\text{-a.s.}$$`

]

If variables `$X_n$` are real-valued, the proposition  follows  from [characterization of weak convergence by simple convergence of quantile functions](#weakconvquantiles)

---

### Proof (for real-valued random variables)

Let `$\Omega= [0,1], \mathcal{F}=\mathcal{B}(\mathbb{R})$` and `$\omega$` be uniformly distributed over `$\Omega = [0,1]$`. Let

`$$Y_n = F_n^\leftarrow(\omega) \text{ and } Y = F^\leftarrow(\omega)$$`

Then for each `$n$`,

`$$P \Big\{ Y_n \leq t \Big\} = P\Big\{\omega : F_n^\leftarrow(\omega) \leq t \Big\} = P\Big\{\omega : \omega \leq F_n(t) \Big\} = F_n(t)$$`

---

### Proof (continued)

As a non-decreasing function has at most countably many discontinuities,

`$$P\Big\{\omega : F^\leftarrow\text{ is continuous at }\omega \Big\}=1$$`

If `$F^{\leftarrow}$` is continuous at `$\omega$`, by the [quantile characterization of weak convergence](weakconvquantiles),

`$$\lim_n F_n^{\leftarrow}(\omega) = F^\leftarrow(\omega)$$`

`$$\text{This translates to }\quad P \Big\{\omega :  \lim_n Y_n(\omega) =Y(\omega) \Big\} = 1$$`

---

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Proposition

Let `$(X_n)_n, X$` be random variables over `$(\Omega, \mathcal{F}, P)$`

If `$(X_n)_n \stackrel{P-\text{a.s.}}{\longrightarrow}  X$`,

then

$$ X_n \rightsquigarrow X$$

]

---
name: secLevyCont
class: middle, inverse, center

## Lévy continuity theorem

---
class: middle, center

---

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Theorem  Lévy's continuity theorem

A sequence `$(P_n)_n$` of probability distributions over `$\mathbb{R}^d$`  converges weakly towards
a probability distribution `$P$` over `$\mathbb{R}^d$`

iff

the sequence of characteristic functions converges pointwise towards the characteristic function of `$P$`.

]

---

To warrant weak convergence of
`$(P_n)_n$` towards `$P$`  it is enough to check that `$\mathbb{E}_{P_n}f \to \mathbb{E}_Pf$`
for functions

`$$f \in \{ \cos(t \cdot), \sin(t \cdot) : t \in \mathbb{R} \}$$`

These functions are bounded and infinitely many times differentiable.

---

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Lemma

Let `$(X_n)_n, X$` and `$Z$` live on the same probability space.

`$$\forall \sigma>0, \quad X_n + \sigma Z \rightsquigarrow X+\sigma Z$$`

then

`$$X_n \rightsquigarrow X$$`

]

---

### Proof

Let `$h$` be bounded by `$1$` and `$1$`-Lipschitz

`$$\begin{array}{rl}\Big| \mathbb{E} h(X_n) -h(X) \Big| & \leq \Big| \mathbb{E} h(X_n) - h(X_n + \sigma Z) \Big| \\ & \quad +  \Big| \mathbb{E} h(X_n + \sigma Z) - h(X + \sigma Z) \Big| \\
& \quad +  \Big| \mathbb{E} h(X + \sigma Z) - h(X) \Big|\end{array}$$`

The first and third summand can be handled in the same way.

---

### Proof (continued)

Let `$\epsilon >0$`,

`$$\begin{array}{rl}\Big| \mathbb{E} h(X_n) - h(X_n + \sigma Z) \Big| & \leq \Big| \mathbb{E} (h(X_n) - h(X_n + \sigma Z)) \mathbb{I}_{\sigma |Z|>\epsilon} \Big| \\ & \qquad + \Big| \mathbb{E} (h(X_n) - h(X_n + \sigma Z)) \mathbb{I}_{\sigma |Z|\leq\epsilon} \Big| \\ & \leq 2 P\{\sigma |Z|>\epsilon\} + \epsilon\end{array}$$`

---

### Proof (continued)

Combining the different bounds leads to

`$$\begin{array}{rl}  \Big| \mathbb{E} h(X_n) -h(X) \Big|  & \leq  4  \underbrace{P\{\sigma |Z|>\epsilon\}}_{(a)} + 2\epsilon + \underbrace{\Big| \mathbb{E} h(X_n + \sigma Z) - h(X + \sigma Z) \Big|}_{(b)}\end{array}$$`

- (b) tends to `$0$` as `$n \uparrow \infty$` ( `$X_n + \sigma Z \rightsquigarrow X+\sigma Z$` )

- (a) tends to `$0$` as `$\sigma \downarrow 0$`

Hence

`$$\limsup_n \Big| \mathbb{E} h(X_n) - h(X_n + \sigma Z) \Big| \leq 2\epsilon$$`

---

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Scheffé's Lemma

Let `$(P_n)_n$`  be a sequence of absolutely continuous probability distributions
with densities `$(f_n)_n$`.

Assume that densities `$(f_n)_n$` converge pointwise towards the
density `$f$` of some probability distribution `$P$`, then

`$$P_n \rightsquigarrow P$$`

]

---

### Proof

`$$\begin{array}{rl} \int_{\mathbb{R}} |f_n(x) - f(x)| \mathrm{d}x  & = \int_{\mathbb{R}} (f(x) - f_n(x))_+  \mathrm{d}x + \int_{\mathbb{R}} (f(x) - f_n(x))_-  \mathrm{d}x \\ & = 2 \int_{\mathbb{R}} (f(x) - f_n(x))_+  \mathrm{d}x\end{array}$$`

Observe

- `$(f - f_n)_+ \leq f$` which belongs to `$\mathcal{L}_1(\mathbb{R}, \mathcal{B}(\mathbb{R}), \text{Lebesgue})$`
- The sequence `$((f - f_n)_+)_n$` converges pointwise to `$0$`

By the dominated convergence theorem, `$\lim_n \int_{\mathbb{R}} |f_n - f| \mathrm{d}x =0$`

For any `$A \in \mathcal{B}(\mathbb{R})$`,

`$$P_n(A) - P(A) = \int_{\mathbb{R}} \mathbb{I}_A (f_n -f) \leq \int_{\mathbb{R}} |f_n-f|$$`

We proved more than weak convergence: `$\lim_n \sup_{A \in \mathcal{B}(\mathbb{R})} |P_n(A) - P(A)| )=0$` <svg aria-hidden="true" role="img" viewBox="0 0 640 512" style="height:1em;width:1.25em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M639.4 433.6c-8.4-20.4-31.8-30.1-52.2-21.6l-22.1 9.2-38.7-101.9c47.9-35 64.8-100.3 34.5-152.8L474.3 16c-8-13.9-25.1-19.7-40-13.6L320 49.8 205.7 2.4c-14.9-6.2-32-.3-40 13.6L79.1 166.5C48.9 219 65.7 284.3 113.6 319.2L74.9 421.1l-22.1-9.2c-20.4-8.5-43.7 1.2-52.2 21.6-1.7 4.1.2 8.8 4.3 10.5l162.3 67.4c4.1 1.7 8.7-.2 10.4-4.3 8.4-20.4-1.2-43.8-21.6-52.3l-22.1-9.2L173.3 342c4.4.5 8.8 1.3 13.1 1.3 51.7 0 99.4-33.1 113.4-85.3l20.2-75.4 20.2 75.4c14 52.2 61.7 85.3 113.4 85.3 4.3 0 8.7-.8 13.1-1.3L506 445.6l-22.1 9.2c-20.4 8.5-30.1 31.9-21.6 52.3 1.7 4.1 6.4 6 10.4 4.3L635.1 444c4-1.7 6-6.3 4.3-10.4zM275.9 162.1l-112.1-46.5 36.5-63.4 94.5 39.2-18.9 70.7zm88.2 0l-18.9-70.7 94.5-39.2 36.5 63.4-112.1 46.5z"/></svg>

---

### Proof of the continuity theorem

Assume the characteristic functions of `$(X_n)_n$` converges pointwise towards the characteristic
function of `$X$`.

Let `$Z$` be a standard Gaussian random variable, independent of all `$(X_n)_n$` and of `$X$`.

For `$\sigma>0$`, the distributions of `$X_n + \sigma Z$` and `$X + \sigma Z$`  have densities
that are uniquely determined by the characteristic functions of `$X_n$` and `$X$`.

By dominated convergence, the densities of `$X_n + \sigma Z$`
converge pointwise towards the density of `$X + \sigma Z$`.

By Scheffé's Lemma,  `$X_n + \sigma Z \rightsquigarrow X + \sigma Z$` for all `$\sigma>0$`

This entails that `$X_n \rightsquigarrow X.$`

---
name: refineLevy
class: middle, center, inverse

## Refining the continuity theorem

---

In some situations, we can prove that a sequence of characteristic functions
converges pointwise towards some function, but we have  no candidate
for the limiting distribution.

The question arises whether the pointwise
limit of characteristic functions is the characteristic function of some probability
distribution or something else.

The answer may be negative: if `$P_n = \mathcal{N}(0, n)$`, the sequence
of characteristic functions is `$\big(t \mapsto \exp(-nt^2/2)\big)_n$` which converges pointwise
to `$0$` except at `$0$` where it is equal to `$1$` all along.

The limit is not the characteristic function of
any probability measure: it is not continuous at `$0$`.

---

The next Theorem settles the question.

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Theorem  Lévy's continuity theorem, second form

A sequence `$(P_n)_n$` of probability distributions over `$\mathbb{R}$`  converges weakly towards
a probability distribution over `$\mathbb{R}$`

iff

the sequence
of characteristic functions converges pointwise towards a function _that is continuous at `$0$`_

The limit function is the characteristic function of some probability distribution.

]

---

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Definition Uniform tightness

A sequence  of Probability measures `$(P_n)_n$`  over `$\mathbb{R}$` is _uniformly
tight_

for every `$\epsilon >0$`,  there exists some compact  `$K \subseteq \mathbb{R}$`
such that

`$$\forall n, \qquad P_n(K) \geq 1 - ϵ$$`

]

---

To establish uniform tightness of `$(P_n)_n$`, it is enough to show that for every `$\epsilon>0$`,
there exists some `$n_0(\epsilon)$`, and some compact  `$K \subseteq \mathbb{R}$` such that

`$$\forall n \geq n_0(\epsilon), \qquad P_n(K) \geq 1 - ϵ$$`

---

We admit the (important) next Theorem

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Theorem Prokhorov-Le Cam

If `$(P_n)_n$` is  a uniformly tight sequence of probability measures on `$\mathbb{R}$`,

then

there is some subsequence `$(P_{n(k)})_{k \in \mathbb{N}}$` such that `$P_{n(k)} ⇝ P$`
for some probability measure `$P$`

]

---
class: middle, center

---
class: middle, center

---

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Uniform tightness Lemma

Let `$(P_n)_n$` be a sequence of probability distributions over `$\mathbb{R}$`,
with characteristic functions `$\widehat{F}_n$`.

If the sequence
`$(\widehat{F}_n)_n$` converges pointwise towards a function that is continuous at `$0$`

then

the sequence `$(P_n)_n$` is _uniformly tight_.

]

---

.pull-left[

The proof of the truncation inequality takes advantage on easy bounds satisfied by the `$\operatorname{sinc}$` function.

`$$\forall t \in \mathbb{R} \setminus[-1,1]$$`

`$$\qquad   \frac{\sin(t)}{t} \leq \sin(1) \leq \frac{6}{7}$$`

]

.pull-right[

]

---

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Truncation Lemma

Assume `$\widehat{F}$` is the characteristic function of some probability measure `$P$` on the real line,

then

`$$\forall u>0, \quad \frac{1}{u}\int_0^u \big(1 - \operatorname{Re}\widehat{F}(v)\big) \mathrm{d}v  \geq \frac{1}{7} P \Big[\frac{-1}{u}, \frac{1}{u}\Big]^c$$`

]

---

### Proof of Truncation Lemma

`$$\begin{array}{rl}\frac{1}{u}\int_0^u \big(1 - \operatorname{Re}\widehat{F}_n(v)\big) \mathrm{d}v & = \frac{1}{u}\int_0^u \Big(\int_{\mathbb{R}} \big(1 - \cos(v w)\big)\mathrm{d}F(w) \Big)\mathrm{d}v \\  & =  \int_{\mathbb{R}}\int_0^u \frac{1}{u} \Big( \big(1 - \cos(v w)\big) \mathrm{d}v  \Big)\mathrm{d}F(w) \\  & =  \int_{\mathbb{R}}  \Big( 1 - \frac{\sin(uw)}{uw}  \Big)\mathrm{d}F(w) \\  & \geq  \int_{|uw| \geq 1}  \Big( 1 - \frac{\sin(uw)}{uw}  \Big)\mathrm{d}F_n(w) \\ & \geq (1- \sin(1)) P \Big[\frac{-1}{u}, \frac{1}{u}\Big]^c \,\end{array}$$`

where the two inequalities follow from the bounds on the `$\operatorname{sinc}$` function.

---

### Proof of Uniform tightness Lemma

Assume that the sequence
`$(\widehat{F}_n)_n$` converge pointwise towards a function `$\widehat{F}$` that is continuous at `$0$`.

Note that `$\widehat{F}_n(0)=1$` for all `$n$`, hence, trivially,  `$1 =\lim_n \widehat{F}_n(0) = \widehat{F}(0)$`.

As `$|\operatorname{Re}\widehat{F}_n(t)|\leq 1$`, `$|\operatorname{Re}\widehat{F}(t)|\leq 1$` also holds.

Fix `$ϵ>0$`, as `$\widehat{F}$` is continuous at `$0$`, for some `$u>0$`, for all `$v \in [-u,u]$`,

`$$0 \leq 1- \operatorname{Re}\widehat{F}(u) \leq \epsilon/2$$`

---

Hence,

`$$0 \leq \frac{1}{u}\int_0^u \big(1 - \operatorname{Re}\widehat{F}(v)\big) \mathrm{d}v \leq \epsilon/2$$`

By dominated convergence,

`$$\lim_n \frac{1}{u}\int_0^u \big(1 - \operatorname{Re}\widehat{F}_n(v)\big) \mathrm{d}v= \frac{1}{u}\int_0^u \big(1 - \operatorname{Re}\widehat{F}(v)\big) \mathrm{d}v  \leq \frac{\epsilon}{2}$$`

For sufficiently large `$n$`, `$0 \leq \frac{1}{u}\int_0^u \big(1 - \operatorname{Re}\widehat{F}_n(v)\big) \mathrm{d}v \leq \epsilon$`.

By the truncation Lemma, for sufficiently large `$n$`

`$$P_n \Big[\frac{-1}{u}, \frac{1}{u}\Big]^c \leq 7 ϵ$$`

`$(P_n)_n$` is uniformly tight

---

We combine the Uniform Tightness Lemma, the Prokhorov-Le Cam Theorem, and the first form
of the Lévy continuity Theorem to establish the (full) Lévy continuity Theorem

### Proof of the second form of the continuity theorem

Under the assumptions of the second form of the continuity  Theorem,

`$\widehat{F}_n \rightarrow F$` with `$F$` continuous at `$0$`

By the Uniform Tightness Lemma, `$(P_n)_n$`  is uniformly tight

By the Prokhorov-Le Cam Theorem, there is a probability measure `$P$` (with characteristic function `$\widehat{F}$`), and a subsequence
`$(P_{n(k)})_{k \in \mathbb{N}}$` such that `$P_{n(k)} \rightsquigarrow P \text{ as } {k \to \infty}$`

---

### Proof of the second form of the continuity theorem (continued)

By the definition of weak convergence,  `$P_{n(k)} \rightsquigarrow P \text{ as } {k \to \infty}$`  entails
`$\widehat{F}_{n(k)} \rightarrow \widehat{F}'' \text{ as } {k \to \infty}$`
where `$\widehat{F}''$` is the characteristic function of `$P$`
--

As `$\widehat{F}_n \rightarrow \widehat{F}$`, by the unicity of limits, `$\widehat{F}= \widehat{F}'$`
and thus `$\widehat{F}$` is the characteristic function the probability distribution `$P$`

Finally, by the first form of the Levy continuity Theorem, `$P_{n}\rightsquigarrow  P \text{ as }{n \to \infty}$`.

<br>

All definitions and results in this section can be extended to the `$k$`-dimensional setting for all `$k \in \mathbb{N}$`

---
class: middle, center, inverse
name: relatconv

## Relations between convergences

---

The alternative characterizations of weak convergence provided by the
Portemanteau Theorem  facilitate  the
proof of the next Proposition.

---

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Proposition

Convergence in probability implies convergence in distribution

If `$(X_n)_n$` is a sequence of RV over `$(\Omega, \mathcal{F}, P)$` such that

`$$X_n \stackrel{P-\text{probability}}{\longrightarrow} X \quad \Rightarrow \quad X_n \rightsquigarrow X$$`

]

---

### Proof

Assume `$(X_n)_n$` converges in probability towards `$X$`.

Let `$h$` be a bounded and Lipschitz function.

Without loss of generality, assume that

`$$|f(x)|\leq 1 \quad \text{and} \quad |f(x)-f(y)|\leq \mathrm{d}(x,y) \qquad \forall x, y$$`

Let `$\epsilon>0$`

`$$\begin{array}{rl}\Big| \mathbb{E}h(X_n) - \mathbb{E}h(X)  \Big| & = \Big| \mathbb{E}\Big[(h(X_n) - h(X) \mathbb{I}_{\mathrm{d}(X,X_n)> \epsilon}\Big] \\ & \qquad  + \mathbb{E}\Big[(h(X_n) - h(X) \mathbb{I}_{\mathrm{d}(X,X_n)\leq \epsilon}\Big] \Big| \\ & \leq  \mathbb{E}\Big[2 \mathbb{I}_{\mathrm{d}(X,X_n)> \epsilon} \Big] \\ \mathbb{E}\Big[ |h(X_n) - h(X)| \mathbb{I}_{\mathrm{d}(X,X_n)\leq \epsilon}\Big] \\ & \leq 2 P\big\{ \mathrm{d}(X,X_n)> \epsilon \big\} + \epsilon\end{array}$$`

---

### Proof (continued)

Convergence in probability entails that

`$$\limsup_n \Big| \mathbb{E}h(X_n) - \mathbb{E}h(X)  \Big| \leq 0$$`

`$$\lim_n \Big| \mathbb{E}h(X_n) - \mathbb{E}h(X)  \Big|=0$$`

This is sufficient to establish convergence in distribution of `$(X_n)_n$`.

---
class: center, middle, inverse
name: clt

## Central limit theorem

---

The Lévy Continuity Theorem
is the conerstone of a very concise proof the simplest version of the
Central Limit Theorem (CLT)

Under square-integrability assumption, the
CLT refines the Laws of Large Numbers.

The CLT states that as `$n$` tends to infinity,
the fluctuations of the empirical mean `$\sum_{i=1}^n X_i/n$` around its expectation
tends to be of order `$1/\sqrt{n}$` and, once rescaled, to be normally distributed.

---

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Theorem  (Central Limit Theorem)

Let `$\ldots, X_n, \ldots$`  be i.i.d. with finite variance `$\sigma^2$`  and expectation `$\mu$`.

Let `$S_n = \sum_{i=1}^n X_i$`.

`$$\frac{1}{\sigma\sqrt{n}} \left({S_n} - n\mu \right) \rightsquigarrow \mathcal{N}\big(0, 1\big)$$`

]

---

### Proof

`$\widehat{F}$` denote the characteristic function of the (common) distribution
of the random variables `$((X_i-\mu)/\sigma)_i$`.

The centering and square integrability assumptions imply that

`$$\widehat{F}(t) = \widehat{F}(0) +\widehat{F}'(0) t + \frac{\widehat{F}^{\prime\prime}(0)}{2} t^2 + t^2 R(t) = 1 - \frac{t^2}{2} + t^2R(t)$$`

where `$\lim_{t \to 0} R(t)=0$`.

---

### Proof (continued)

Let `$\widehat{F}_n$` denote the characteristic function of `$\frac{1}{\sigma\sqrt{n}} \left({S_n} - n\mu \right)$`.

Fix `$t \in \mathbb{R}$`,

`$$\widehat{F}_n(t) = \Big(\widehat{F}(t/\sqrt{n})\Big)^n= \Big(1 - \frac{t^2}{2n} + \frac{t^2}{n} R(t/\sqrt{n})\Big)^n$$`

As `$n\to \infty$`,

`$$\lim_n \Big(1 - \frac{t^2}{2n} + \frac{t^2}{n} R(t/\sqrt{n})\Big)^n  = \mathrm{e}^{- \frac{t^2}{2}}$$`

---

The conditions in the Theorem statement allows for a short proof

They are by no mean necessary

- The summands need not be identically distributed.

- The summands need not be independent.

A version of the Lindeberg-Feller Theorem  states that under mild assumptions, centered and normalized sums of independent square-integrable random variables converge in distribution towards a Gaussian distribution.

Consider the number `$Z_n$` of cycles in a random permutation over `$1, \ldots, n$`

`$$Z_n \sim \sum_{i=1}^n Y_i \qquad \text{with}\quad Y_i \sim \operatorname{Be}(1/i)$$`

and

`$$\frac{1}{\sqrt{H_n}}\left(Z_n - H_n\right) ⇝ \mathcal{N}(0,1)$$`

with `$H_n = \sum_{i=1}^n 1/i$`

---

### [De Moivre](https://en.wikipedia.org/wiki/De_Moivre–Laplace_theorem) CLT illustrated

.pull-left[

]

.pull-right[

Pointwise convergence of CDFs towards the Gaussian CDF of `$\mathcal{N}(0,1)$` (plain line)

The dotted and the dashed lines represent the CDF of

`$$\frac{(X_n -np)}{\sqrt{n(p*(1-p))}}$$`

where

`$$X_n \sim \text{Binom}(n,p)$$`

for `$p=.3$` and `$n=30 \text(dotted), 100 \text(dashed)$`.

]

---
name: cramerwolddevice
class: center, middle, inverse

## Cramer-Wold device

---

So far, we have discussed characteristic functions for real valued random variables.

Characteristic functions can also be defined for vector-valued random variables.

If `$X$` is a `$\mathbb{R}^k$`-valued random variable,
its characteristic function maps `$\mathbb{R}^k$` to `$\mathbb{C}$`

`$$\begin{array}{rl}\mathbb{R}^k & \to \mathbb{C} \\ t & \mapsto \mathbb{E}\mathrm{e}^{i \langle t, X\rangle}\end{array}$$`

---

The importance of multivariate characteristic functions is reflected
in the  next device which proof is left to the reader.

It consists in the adapting the
proof of the injectivity Theorem

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Theorem Cramer-Wold device

The distribution of a `$\mathbb{R}^k$`-valued random vector `$X = (X_1, \ldots, X_k)^T$` is completely determined by the collection of distributions of univariate random variables

`$$\langle t, X\rangle =\sum_{i=1}^n t_i X_i  \text{ where } (t_1, \ldots, t_n)^T \in \mathbb{R}^n$$`

]

---

The Cramer-Wold device  provides a short path to the Multivariate Central Limit Theorem

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Theorem

Let `$X_1, \ldots, X_n, \ldots$`  be i.i.d. vector valued random variables with finite covariance `$\Gamma$`  and expectation `$\mu$`.

Let `$S_n = \sum_{i=1}^n X_i$`.

`$$\sqrt{n} \left(\frac{S_n}{n} - \mu \right) \rightsquigarrow \mathcal{N}\big(0, \Gamma\big)$$`

]

---
name: weakconvtransforms
class: center, middle, inverse

## Weak convergence and transforms

---

.pull-left[
We introduced different characterizations of probability distributions:

- probability generating functions,
- Laplace transforms,
- Fourier transforms (characteristic functions),
- cumulative distribution functions,
- quantiles functions.

]

.pull-right[

Within their scope, all those transforms are _convergence determining_: if a sequence of probability distributions converges weakly, so does (pointwise) the corresponding sequence of transforms, at least at the continuity points of the limiting transform.

In the next two theorems, each random variable is assumed to live on
some (implicit) probability space.

]

---

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Theorem

A sequence of _non-negative_ random variables  `$(X_n)_n$`  converges in distribution
towards the _non negative_ random variable  `$X$` iff the sequence of Laplace transforms converges pointwise towards the Laplace transform of the probability distribution of `$X$`.

]

The proof parallels the derivation of the continuity Theorem

---

As probability generating functions allows us to recover Laplace transforms, the next theorem is a special case of the statement concerning Laplace transforms.

.bg-light-gray.b--light-gray.ba.bw1.br3.shadow-5.ph4.mt5[

### Theorem

A sequence of integer-valued random variables  `$(X_n)_n$`  converges in distribution
towards the integer-valued random variable  `$X$`

iff

the sequence of Laplace transforms `$\lambda \mapsto \mathbb{E} \mathrm{e}^{- \lambda X_n}, \lambda\geq 0$` converges _pointwise_ towards the Laplace transform `$\lambda \mapsto \mathbb{E} \mathrm{e}^{- \lambda X}$` of the
probability distribution of `$X$`.

]

---
exclude: true

### Bibliographic remarks

@MR1932358 discusses convergence in distributions in two chapters: the first one is dedicated to distributions on `$\mathbb{R}^d$` and the central limit theorem;
the second chapter addresses more general universes. In the first chapter, the central limit theorem is extended to triangular arrays
that is to sequences of not necessarily identically distributed random variables (Lindeberg's Theorem).

@MR1932358 investigates convergence in distributions as _convergence of laws on separable metric spaces_, that is in a much
broader context than we do in these notes. The reader will find there a complete proof of the Prokhorov-Le Cam Theorem
and an in-depth discussion of its corollaries. In [@MR1932358], a great deal of effort is dedicated to the metrization
of the weak convergence topology. The reader will also find in this book a full picture of almost sure representation arguments.

The proof of the Lévy Continuity Theorem given here is taken from [@MR1873379].

Using metrizations for weak convergence allows us to investigate rate of convergence in limit theorems.
This goes back at least to the Berry-Esseen's Theorem (1942). Quantitative approaches to weak convergence
have acquired a new momentum with the popularization of Stein's method.
This methods is geared towards,  but exclusively focused on,  general yet  quantitative versions of the Central Limit Theorem [@MR2732624] .
A thorough yet readable introduction to Stein's method is [@MR2861132].

---
exclude: true

### References

.pull-left[

]

.pull-right[

]

---

### References

[Dudley, Richard M.(1-MIT)](https://news.mit.edu/2020/richard-dudley-mit-mathematics-professor-emeritus-dies-0218)

__Real analysis and probability.__

The Wadsworth & Brooks/Cole Mathematics Series. 1989. xii+436 pp. ISBN: 0-534-10050-3

> This is a remarkable textbook on real analysis and probability.

> ...

> Among the less standard topics contained in the book, the following should be mentioned: weak convergence of probability measures on metric spaces, with the Strassen and Kantorovich-Rubinstein theorems, and the Skorokhod characterization of weak convergence;

> ...

From  [MR0982264  Math Reviews by E. Giné](https://mathscinet-ams-org.ezproxy.math-info-paris.cnrs.fr/mathscinet-getitem?mr=982264)

---

class: middle, center, inverse

background-image: url('./img/pexels-cottonbro-3171837.jpg')
background-size: cover

# The End