A symmetric matrix $A \in \R^{n \times n}$ has orthogonal eigenvectors and real eigenvalues. Moreover, it can be orthogonally diagonalized into an orthogonal matrix $Q$, containing its eigenvectors, and a diagonal matrix $\Lambda$, containing its eigenvalues.
$$ A = Q\Lambda Q^T $$
Symmetric matrices $A \in \R^{n \times n}$ are characterized by the sign of their eigenvalues which, equivalently, corresponds to the sign of the product $x^T A x$.
A smooth function $f : \R^n \rightarrow \R^m$ can be approximated to an arbitrary degree via its Taylor expansion centered at a point $x_0 \in \R^n$. In this course, we typically look at first and second order approximations, which only require $1$ and $2$ times continuous differentiability.
$$ f(x) \approx f(x_0) + \nabla f(x_0)^T(x-x_0) + O(d^2) \\
f(x) \approx f(x_0) +\nabla f(x_0)^T(x-x_0) + \frac{1}{2}(x-x_0)^T\nabla^2 f(x_0)(x-x_0)+O(d^3) $$
<aside> <img src="/icons/castle_yellow.svg" alt="/icons/castle_yellow.svg" width="40px" />
Mid-Value Theorem. States that, for any $x, x_0 \in \R^n$ there exist points $u, v \in \R^n$ on the straight line segment from $x$ to $x_0$ which make the first and second order Taylor approximations exact.
$$ f(x) = f(x_0) + \nabla f(u)^T(x-x_0) $$
$$ f(x) = f(x_0 ) + \nabla f(x_0)^T(x-x_0) + \frac{1}{2}(x-x_0)^T \nabla^2f(v)(x-x_0) $$
The point $u$ can be interpreted as the point for which the total change in $f$ between $x$ and $x_0$ is exactly a linear term in $u$. Analogously, the point $v$ can be interpreted as the point for which the total change in $f$ between $x$ and $x_0$ can be written as a linear approximation and a quadratic term in $v$.
</aside>