Linear algebra for covariance and hedging

Definition All expectations on this page are under one fixed probability measure, for example the physical measure $\mathbb{P}$. Let $R = (R_1, \dots, R_d)^{\top}$ be a random vector, such as the gains of $d$ instruments over one period, with $E[R_i^2] < \infty$ for every $i$, and let $\mu = E[R]$. Its covariance matrix is the $d \times d$ matrix

with the expectation taken entry by entry, so that $\Sigma_{ij} = \operatorname{Cov}(R_i, R_j)$ and $\Sigma_{ii} = \operatorname{Var}(R_i)$. The finite second moments make every entry finite, since $\lvert \operatorname{Cov}(R_i, R_j) \rvert \le \sqrt{\operatorname{Var}(R_i) \operatorname{Var}(R_j)}$. Each entry is the expectation of a product of two scalar deviations, which commute, so $\Sigma_{ij} = \Sigma_{ji}$ and $\Sigma$ is symmetric. This is the population covariance of the model. The sample covariance of observations $r_1, \dots, r_n$ with mean $\bar{r}$,

is an estimate of it and a different object. Its rank is at most $n - 1$, so it is singular whenever $n \le d$.

Definition A real symmetric $d \times d$ matrix $A$ is positive semidefinite if $w^{\top} A w \ge 0$ for every $w \in \mathbb{R}^d$, and positive definite if $w^{\top} A w > 0$ for every $w \ne 0$. Every covariance matrix is positive semidefinite. For fixed weights $w$, the portfolio $w^{\top} R$ has the centered value $w^{\top}(R - \mu)$, and

where the middle equality is linearity of expectation applied to the double sum $\sum_{i,j} w_i w_j (R_i - \mu_i)(R_j - \mu_j)$, which is the square of $w^{\top}(R - \mu)$. The matrix $\Sigma$ is positive definite exactly when no nonzero $w$ makes $w^{\top} R$ equal to a constant with probability $1$.

Definition A number $\lambda$ is an eigenvalue of a square matrix $A$, with eigenvector $q \ne 0$, if $Aq = \lambda q$. The spectral theorem for real symmetric matrices, stated here without proof, says that a real symmetric $d \times d$ matrix $A$ has real eigenvalues $\lambda_1, \dots, \lambda_d$, listed with multiplicity, and an orthogonal matrix $Q$, with $Q^{\top} Q = I$, whose columns $q_1, \dots, q_d$ are corresponding unit eigenvectors, so that

Then for every $w \in \mathbb{R}^d$,

so $A$ is positive semidefinite exactly when every $\lambda_j \ge 0$, and positive definite exactly when every $\lambda_j > 0$. For a covariance matrix, $\lambda_j = q_j^{\top} \Sigma q_j$ is the variance of the portfolio with holding vector $q_j$. An eigenvalue $0$ means that $q_j^{\top}(R - \mu) = 0$ with probability $1$, a combination that carries no risk in this model. That is a statement about the model’s random vector, not about every source of risk in a real position.

Definition Let $X$ and $Y_1, \dots, Y_m$ be random variables with mean zero and finite second moments, and use the inner product $\langle U, W \rangle = E[UW]$, which for centered variables is $\operatorname{Cov}(U, W)$. The hedge span is the set of payoffs $h^{\top} Y = \sum_i h_i Y_i$ with $h \in \mathbb{R}^m$. A payoff $P$ in the span is the orthogonal projection of $X$ onto the span if the residual $X - P$ is orthogonal to every $Y_i$, that is, $E[(X - P) Y_i] = 0$ for $i = 1, \dots, m$. Then the residual is orthogonal to every payoff $d$ in the span, and

because the cross term $-2E[(X - P)d]$ vanishes. So $P$ has the smallest mean squared error among all payoffs in the span, and any other payoff $P + d$ with the same error has $E[d^2] = 0$, so it equals $P$ with probability $1$. A projection always exists, as the pseudoinverse card shows, even when the holdings that produce it are not unique.

Definition Let a target gain $X$ and hedge gains $Y = (Y_1, \dots, Y_m)^{\top}$ be centered with finite second moments, let $\Sigma_Y$ be the covariance matrix of $Y$ and let $c = \operatorname{Cov}(Y, X)$ be the vector with entries $\operatorname{Cov}(Y_i, X)$. A minimum-error hedge is a holding vector $h$ that minimizes

where the second line comes from squaring the whole residual and taking expectations term by term. The gradient $\nabla J(h) = -2c + 2\Sigma_Y h$ vanishes exactly at the solutions of the normal equations

Since $J$ is differentiable, every minimizer solves them. Conversely, the normal equations say $\operatorname{Cov}(Y_i, X - h^{\top} Y) = 0$ for every $i$, so by the projection card every solution is a minimizer. For any solution the minimum residual variance is $J(h) = \operatorname{Var}(X) - h^{\top} c$. If $\Sigma_Y$ is invertible, the hedge is unique and equals $\Sigma_Y^{-1} c$. Centering removes the means, which are handled separately, and the result concerns this squared-error criterion only.

Definition Let $\Sigma = Q \Lambda Q^{\top}$ be a spectral decomposition of a real symmetric positive semidefinite matrix. Its Moore-Penrose pseudoinverse is

where $\lambda^{+} = 1/\lambda$ for $\lambda > 0$ and $\lambda^{+} = 0$ for $\lambda = 0$. It equals $\Sigma^{-1}$ when $\Sigma$ is invertible. For a general matrix it is defined through the singular value decomposition, which is not derived here. If $c$ lies in the range of $\Sigma$, then $h^{+} = \Sigma^{+} c$ solves $\Sigma h = c$, the solutions are exactly the vectors $h^{+} + n$ with $\Sigma n = 0$, and $h^{+}$ is the solution with the smallest Euclidean norm, because it lies in the range and so is orthogonal to every null vector $n$. In a hedge with centered gains $Y$, a null vector $n$ of $\Sigma_Y$ has $\operatorname{Var}(n^{\top} Y) = n^{\top} \Sigma_Y n = 0$, so $n^{\top} Y = 0$ with probability $1$. Hence all solutions give the same hedge payoff and the same residual variance, and $n^{\top} c = \operatorname{Cov}(n^{\top} Y, X) = 0$. The covariance vector of a genuine square-integrable target is therefore orthogonal to the null space, which for a symmetric matrix means that it lies in the range, so the normal equations always have a solution.