Outer Product

Constructs a linear transformation from $\mathbf{u} \in \R^{m}$ and $\mathbf{v} \in \R^n$ which maps an input $\mathbf{x} \in \R^n$ to a vector $\mathbf{y} \in \text{span}\{\mathbf{u}\}$ with norm proportional to the amount $\mathbf{x}$ points in the direction of $\mathbf{v}$.

$$ \mathbf{u}\mathbf{v}^T= \mathbf{u} \begin{bmatrix} v_1 \hspace{3pt} \dots \hspace{3pt} v_n \end{bmatrix} = \begin{bmatrix} v_1 \mathbf{u} \hspace{5pt} v_2\mathbf{u} \hspace{5pt} \dots \hspace{5pt} v_n\mathbf{u}\end{bmatrix} \\\hspace{0pt} \\

(\mathbf{u}\mathbf{v}^T)\mathbf{x} = (\mathbf{v}^T\mathbf{x})\mathbf{u} $$

Note that because each column is a scalar multiple of $\mathbf{u}$, any outer product is of rank $1$.


Matrices

Rectangular arrays of entries with size equal to the total number of entries.

$$ A = \begin{bmatrix} a_{11} & a_{12} & \dots & a_{1n} \\ a_{21} & a_{22} & \dots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} \end{bmatrix} $$

$$ A\in \R^{m \times n} : \begin{cases} \text{ Tall } \hspace{27pt} m > n \\ \text{ Square } \hspace {15pt} m=n \\ \text{ Wide } \hspace {22pt} m < n\end{cases} \\

$$

Matrices can also be thought of as a collection of column or row vectors.

$$ A = \begin{bmatrix} a_{11} & a_{12} & \dots & a_{1n} \\ a_{21} & a_{22} & \dots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} \end{bmatrix} = \begin{bmatrix} \begin{bmatrix} a_{11} \\ a_{21} \\ \vdots \\ a_{m1} \end{bmatrix} \begin{bmatrix} a_{12} \\ a_{22} \\ \vdots \\ a_{m2}\end{bmatrix} \dots \begin{bmatrix} a_{1n} \\ a_{2n} \\ \vdots \\ a_{mn} \end{bmatrix} \end{bmatrix} = \begin{bmatrix} \mathbf{a}_1 \hspace{2pt} \mathbf{a}_2 \hspace{2pt} \dots \hspace{2pt} \mathbf{a}_n\end{bmatrix} $$

$$ A = \begin{bmatrix} a_{11} & a_{12} & \dots & a_{1n} \\ a_{21} & a_{22} & \dots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} \end{bmatrix} = \begin{bmatrix} \begin{bmatrix} a_{11} \hspace{5pt} a_{12} \hspace{5pt} \dots \hspace{5pt} a_{1n} \end{bmatrix} \\ \begin{bmatrix} a_{21} \hspace{5pt} a_{22} \hspace{5pt} \dots \hspace{5pt} a_{2n} \end{bmatrix} \\ \vdots \\ \begin{bmatrix} a_{m1} \hspace{2pt} a_{m2} \hspace{3pt} \dots \hspace{2pt} a_{mn} \end{bmatrix} \end{bmatrix} = \begin{bmatrix} \boldsymbol{\alpha}_1 ^{\text{T}}\\ \boldsymbol{\alpha}_2^{\text{T}} \\ \vdots \\ \boldsymbol{\alpha}_m^{\text{T}}\end{bmatrix} $$

Matrix Equality

2 matrices of the same size are equal if and only if their entries are equal at every index.

$$ A, B \in \R^{m \times n}, A = B \hspace{15pt} \leftrightarrow \hspace{15pt} \forall i \in \{1 \space \dots \space m \}, \forall j \in \{1 \space \dots \space n \}, a_{ij} = b_{ij} $$

Matrix Transpose

The transpose of a matrix $A \in \R^{m \times n}$ is the matrix $A^{\text{T}}\in \R^{n \times m}$ whose columns are the rows of $A$ and vice versa.

$$ A = \begin{bmatrix} a_{11} & a_{12} & \dots & a_{1n} \\ a_{21} & a_{22} & \dots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} \end{bmatrix} \rightarrow A^{\text{T}} = \begin{bmatrix} a_{11} & a_{21} & \dots & a_{m1} \\ a_{12} & a_{22} & \dots & a_{m2} \\ \vdots & \vdots & \ddots & \vdots \\ a_{1n} & a_{2n} & \dots & a_{mn} \end{bmatrix} \\ a^{\text{T}}{ij} = a{ji}: ij^{\text{th}} \text{ entry of } A^{\text{T}} \text{ is } ji^{\text{th}} \text{ entry of } A $$