Distance

The Euclidean distance between two vectors $\mathbf{x} , \mathbf{y}$ is the $L_2$ norm of their difference.

$$ \text{dist}(\mathbf{x}, \mathbf{y}) = ||\mathbf{x} - \mathbf{y}||_2 $$

• K-Nearest Neighbors

Inner Product

An function $\langle \cdot \space , \cdot \rangle : V \times V \rightarrow F$ is an inner product if and only if it has following 4 properties.

1. Distributivity. $\langle \mathbf{x} \space , \mathbf{y + z} \rangle = \langle \mathbf{x} \space , \mathbf{y} \rangle + \langle \mathbf{x} \space , \mathbf{z} \rangle$
2. Scaling. $\langle \alpha \mathbf{x} \space , \mathbf{y} \rangle = \alpha \langle \mathbf{x} \space , \mathbf{y} \rangle$
3. Positive Definiteness. $\langle \mathbf{x} \space , \mathbf{x} \rangle \geq 0 \text{ and } \langle \mathbf{x} \space , \mathbf{x} \rangle = 0 \leftrightarrow \mathbf{x} = \mathbf{0}$

• Properties for $F = \R$
• Properties for $F = \mathbb{C}$

Canonical Inner Products

The canonical inner product varies depending on the field associated with its vector space.

$$ \langle \mathbf{x} \space , \mathbf{y} \rangle = \mathbf{x}^T \mathbf{y} = \sum _{i=1} ^n x_i y_i $$

For real vector spaces.

$$ \langle \mathbf{x} \space , \mathbf{y} \rangle = \mathbf{x}^{\text{T}} \mathbf{y}^* = \mathbf{x}^{\text{H}} \mathbf{y} = \sum _{i=1} ^n x_i y_i^* $$

For complex vector spaces (we alter the definition for real vector spaces to ensure the result means the same thing).

Average

The inner product can be used to compute the average entry of a vector.

$$ \text{avg}(\mathbf{x}) = \frac{1}{n} \langle \mathbf{1} \space , \mathbf{x} \rangle $$