Cauchy-Schwarz and Orthonormal Expansions

Definition For all vectors $x$ and $y$ in an inner product space $V$ over $\mathbb{F}$,

No dimension bound is needed. If $x = 0$, both sides are $0$. If $x \ne 0$, write $h = \langle x, y \rangle$ and $d = \lVert x \rVert^2$, which is positive by positive definiteness. Set $c = h / d$ and $r = y - cx$. Then $\langle x, r \rangle = h - cd = 0$, and expanding with $\langle y, x \rangle = \overline{h}$ gives

because the products $c\,\overline{h}$, $\overline{c}\,h$ and $\lvert c \rvert^2 d$ all equal $\lvert h \rvert^2 / d$. Since $\lVert r \rVert^2 \ge 0$, multiplying by $d$ gives $\lvert h \rvert^2 \le \lVert x \rVert^2 \lVert y \rVert^2$, and taking nonnegative square roots gives the inequality. The coefficient must be $\langle x, y \rangle / \lVert x \rVert^2$ with $x$ in the first slot. Its conjugate would generally leave a remainder that is not orthogonal to $x$.

Definition For vectors $x$ and $y$ in an inner product space over $\mathbb{F}$,

holds if and only if $x$ and $y$ are linearly dependent over $\mathbb{F}$. If $x = 0$, both sides are $0$ and the pair is dependent. If $x \ne 0$, equality means $\lVert r \rVert^2 = 0$ in the proof above, and positive definiteness turns this into $r = 0$, that is $y = cx$ with $c = \langle x, y \rangle / \lVert x \rVert^2$. Conversely, if $x \ne 0$ and the pair is dependent, then $y = ax$ for some scalar $a$, and $\lvert \langle x, ax \rangle \rvert = \lvert a \rvert \, \lVert x \rVert^2 = \lVert x \rVert \, \lVert ax \rVert$.

Definition For all vectors $x$ and $y$ in an inner product space over $\mathbb{F}$,

Using $\operatorname{Re} z \le \lvert z \rvert$ and then Cauchy-Schwarz,

and both sides of the inequality are nonnegative, so square roots may be taken. Equality forces both steps to be equalities, so the pair is dependent and $\langle x, y \rangle$ is a nonnegative real number. This happens exactly when one of the vectors is a nonnegative real multiple of the other. Together with $\lVert ax \rVert = \lvert a \rvert \, \lVert x \rVert$ and positivity, this shows that $\lVert x \rVert = \sqrt{\langle x, x \rangle}$ is a norm, and the distance $\lVert x - y \rVert$ satisfies $\lVert x - z \rVert \le \lVert x - y \rVert + \lVert y - z \rVert$.

Definition Let $a_1, \dots, a_n$ be linearly independent vectors in an inner product space $V$ over $\mathbb{F}$. Set $q_1 = a_1 / \lVert a_1 \rVert$ and, for $k = 2, \dots, n$,

Then $(q_1, \dots, q_n)$ is orthonormal, meaning $\langle q_i, q_j \rangle$ is $1$ for $i = j$ and $0$ otherwise, and $\operatorname{span}(q_1, \dots, q_k) = \operatorname{span}(a_1, \dots, a_k)$ for every $k$. Each $r_k$ is orthogonal to every earlier $q_i$, because linearity in the second slot and the orthonormality of $q_1, \dots, q_{k-1}$ give $\langle q_i, r_k \rangle = \langle q_i, a_k \rangle - \langle q_i, a_k \rangle = 0$. Each $r_k$ is nonzero, since $r_k = 0$ would put $a_k$ in $\operatorname{span}(q_1, \dots, q_{k-1}) = \operatorname{span}(a_1, \dots, a_{k-1})$. Rearranging the subtraction writes $a_k$ as a combination of $q_1, \dots, q_k$, which keeps the spans equal at each step. If the inputs are dependent, the first input that lies in the span of the earlier ones leaves a zero remainder, or is zero itself when it is $a_1$. That zero signals the dependence, and it cannot be normalized. Applied to any basis, the process shows that every finite-dimensional inner product space has an orthonormal basis.

Definition Let $q_1, \dots, q_n$ be an orthonormal basis of a finite-dimensional inner product space $V$ over $\mathbb{F}$. Every $v \in V$ has the orthonormal expansion

Indeed, if $v = \sum_k c_k q_k$, pairing with $q_j$ in the first slot removes every term except the $j$th, so $\langle q_j, v \rangle = c_j$. For any finite orthonormal list and any scalars $c_j$, the cross terms $\overline{c_i}\,c_j \langle q_i, q_j \rangle$ with $i \ne j$ vanish, which gives the Pythagorean identity

Applied to the expansion of $v$, it gives Parseval’s identity

and the same expansion gives $\langle v, w \rangle = \sum_j \overline{\langle q_j, v \rangle}\,\langle q_j, w \rangle$ for all $v, w \in V$. The coefficient is $\langle q_j, v \rangle$. The reversed pairing $\langle v, q_j \rangle$ is its conjugate.

Definition Let $q_1, \dots, q_k$ be a finite orthonormal list in an inner product space $V$ over $\mathbb{F}$, not necessarily a basis. For every $v \in V$,

with equality if and only if $v \in \operatorname{span}(q_1, \dots, q_k)$. To prove it, let $p = \sum_j \langle q_j, v \rangle \, q_j$ and $r = v - p$. Then $\langle q_j, r \rangle = 0$ for every $j$, so $\langle p, r \rangle = 0$ and

Equality means $r = 0$, so $v = p$ lies in the span. Conversely, a vector in the span equals its own expansion, so $r = 0$. When the list is an orthonormal basis of a finite-dimensional $V$, every $v$ lies in the span and Bessel’s inequality becomes Parseval’s identity.