Eigen Decomposition - Building the Covariance Matrix

For a Covariance matrix $C_X$, we want to find:

$$C_X=VE{V}^T$$

Where:

• $E$ is a diagonal matrix containing the eigenvalues
• $V$ is a matrix whose columns are the corresponding eigenvectors

Steps

1. Finding eigenvalues: We solve the characteristic equation $\det \left(C_X-\lambda I\right)=0$ to find values of $\lambda$ (eigenvalues).
2. Finding eigenvectors: For each eigenvalue ${\lambda }_i$, we solve $\left(C_X-{\lambda }_{i}I\right)v_i=0$ to find the corresponding eigenvector $v_i$.
3. Building matrices:

• $E$ is a diagonal matrix with eigenvalues ${\lambda }_i$ on the diagonal
• $V$ is a matrix with eigenvectors $v_i$ as columns