Convergence describes the tendency of a sequence to approach a fixed point known as its limit.
The convergence of a sequence $\{a_n\}$ of real numbers is defined with no reference to probability.
$$ \lim _{n \rightarrow \infty } a_n = a \iff \forall \varepsilon >0, \exists N\in \N : \forall n \geq N, |a_n -a| \leq \varepsilon $$
The convergence of a sequence $\{X_n\}$ of random variables, with CDFs $\{F_n\}$, to a random variable $X$, with CDF $F$, in distribution is defined to occur if the CDF sequence is limited by $F$.
$$ X_n \xrightarrow{\mathcal{D}} X \iff \lim _{n\rightarrow \infty} F_n(x) = F(x) $$
The convergence of a sequence $\{X_n\}$ of random variables in probability is defined to occur if the probability of observing anything but its limit $X$ grows smaller as the sequence progresses.
$$ X_n \xrightarrow{p} X \iff \forall \varepsilon >0, \lim _{n\rightarrow \infty} P(|X_n -X| \geq \varepsilon) = 0 $$
This is a weaker notion than almost sure convergence, since it doesn’t make a statement about the sequence converging to $X$ itself.
The almost sure convergence of a sequence $\{X_n\}$ of random variables is defined to occur if the sequence is limited by $X$ with probability $1$.
$$ X_n \xrightarrow{a.s.} X \iff P(\lim _{n\rightarrow \infty} X_n = X) = 1\equiv P(\{\omega \in \Omega : \lim _{n\rightarrow \infty} X_n (\omega) = X(\omega)\}) = 1 $$
This is a stronger notion that almost sure convergence, since it it says that $X_n$ is almost always $X$ as the sequence progresses.
<aside> <img src="/icons/castle_yellow.svg" alt="/icons/castle_yellow.svg" width="40px" />
Borel-Cantelli Lemmas. Given a sequence $\{A_n\}$ of events in a probability space, we say $A_n$ occurs infinitely often ($\text{i.o.}$) if there are infinitely many indices for which $A_n$ occurs. The lemmas are results on when $A_n$ occurs infinitely often:
$$ \mathbf{1. } \space \sum _{n=1} ^\infty P(A_n ) \text{ is finite} \implies P(A_n \space \text{i.o.}) = 0 \hspace{93pt} \\
\mathbf{2. } \space A_1 \perp A_2 \perp \dots \text{ and } \sum _{n=1}^\infty P(A_n) \text{ is infinite } \implies P(A_n \text{ i.o.}) = 1 $$
Moreover, we define $\{A_n \space i.o.\} := \cap _{n \geq 1} \cup _{m \geq n } A_n$.
</aside>
<aside> <img src="/icons/castle_yellow.svg" alt="/icons/castle_yellow.svg" width="40px" />
Almost Sure Convergence by Borel-Cantelli Characterization. A sequence $\{X_n\}$ converges almost surely to $X$ iff for every $\varepsilon >0$, the event $A_n (\varepsilon) := \{ |X_n - X| > \varepsilon\}$ occurs only finitely often almost surely.
$$ X_n \xrightarrow{a.s.} X \iff \forall \varepsilon > 0, \space P(A_n(\varepsilon) \space i.o.) = 0 $$