Terminology

• Sample Spaces contain outcomes, which can be thought of as arbitrary encodings of information about the outcomes of a random experiment.
• Random variables are quantities which measure outcomes that arise in uncertainty; formally, they are functions which assign numerical or categorical values to samples.
• Observed variables, also known as evidence, are quantities which are known or given.
• Unobserved variables are quantities which are unknown.
• Models relate unobserved and observed variables.
• Assignments, also known as outcomes, ****are a selection of outcomes amongst those variables. Formally, they are disjoint events or partitions of the sample space defined.
• Normalization of a distribution is the process of making “probabilities” in it sum to 1.
• Marginalization of a joint distribution is the process of producing a marginal distribution; i.e. marginalizing out uncertainty of/dependence on other variables.

Probability

Probability distributions assign weight to outcomes in the sample space. Probability, then, is just about the weighted proportion of the sample space taken up by a particular outcome.

Probability Space. For random variables $X_1$ and $X_2$ with domains $\{+x_1, -x_1 \}$ and $\{+x_2, -x_2 \}$ respectively, the sample space can be thought of as partitioned into outcomes of these random variables. The probability distribution $\Pr$ then weights each of these partitions.

Probability Distributions

An unobserved random variable $X$ has probability assigned to each of its outcomes $x_i$ through a probability distribution $\Pr[X=x_i]$, commonly notated simply as $\Pr[X]$.

$$ \forall x_i, \space \Pr[X = x_i] \geq 0 \hspace{10pt} \text{ and } \hspace{10pt} \sum _{i}\Pr[X=x_i] = 1 $$

Joint Distribution