Eigenvalues and Cayley-Hamilton

Take the matrix

and multiply it by the vector \mathbf{v} = \begin{bmatrix} 1 \\ 0 \end{bmatrix}:

The output is 3 times the input. The matrix A did not rotate \mathbf{v} or tilt it sideways — it just stretched it by a factor of 3, like pulling a rubber band along its own direction.

Now try \mathbf{w} = \begin{bmatrix} 1 \\ -1 \end{bmatrix}:

Again, the output is a scalar multiple of the input — this time by 2.

Most vectors get both rotated and stretched when you multiply them by a matrix. But these two — \mathbf{v} and \mathbf{w} — only get stretched (or compressed). They stay on the same line they started on. Vectors with this special property are called eigenvectors, and the scale factors (3 and 2) are called eigenvalues.

These numbers are not curiosities. The eigenvalues of a matrix encode the most fundamental information about what the matrix does — how much it stretches, whether it flips directions, whether it is stable or unstable in applications from differential equations to Google's PageRank algorithm to the vibration modes of a bridge. Finding them is one of the central problems in linear algebra.

The equation A\mathbf{v} = \lambda\mathbf{v}

Two things to note.

The eigenvector must be nonzero. The zero vector trivially satisfies A\mathbf{0} = \lambda\mathbf{0} for every \lambda, so allowing it would make every number an eigenvalue, which is useless.

The eigenvalue can be zero. If \lambda = 0, the equation says A\mathbf{v} = \mathbf{0} — the matrix sends some nonzero vector to zero. This happens exactly when A is singular (non-invertible), which means \det A = 0. So zero is an eigenvalue of A if and only if A is not invertible.

Finding eigenvalues: the characteristic equation

How do you find the eigenvalues without guessing eigenvectors? Rearrange the defining equation:

This is a homogeneous system (A - \lambda I)\mathbf{v} = \mathbf{0}, and you need a nonzero solution \mathbf{v}. From the consistency of systems article, a homogeneous system has non-trivial solutions exactly when \text{rank}(A - \lambda I) < n, which for a square matrix means

This is the key equation.

Computing the characteristic equation for a 2 \times 2 matrix

Take A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}. Then

Expanding:

where \text{tr}(A) = a + d is the trace (sum of diagonal entries) and \det(A) = ad - bc is the determinant. So for any 2 \times 2 matrix, the eigenvalues satisfy the quadratic

This is a clean formula worth remembering. The sum of the eigenvalues equals the trace, and the product of the eigenvalues equals the determinant.

Computing the characteristic equation for a 3 \times 3 matrix

For A = \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{bmatrix}, the characteristic polynomial is a cubic:

In practice, you compute A - \lambda I explicitly and expand the determinant along a row or column, just as you would for any 3 \times 3 determinant. The result is a cubic equation in \lambda, which you solve by inspection, factoring, or the rational root theorem.

Worked examples

Properties of eigenvalues

Several useful facts follow directly from the characteristic equation.

1. The sum of eigenvalues equals the trace. For any n \times n matrix, \lambda_1 + \lambda_2 + \cdots + \lambda_n = \text{tr}(A). You already verified this in both examples.

2. The product of eigenvalues equals the determinant. \lambda_1 \cdot \lambda_2 \cdots \lambda_n = \det(A). This is because the constant term of the characteristic polynomial is \det(A) (set \lambda = 0), and the product of the roots of a monic polynomial is (-1)^n times the constant term.

3. A matrix is singular if and only if zero is an eigenvalue. This is a direct consequence of property 2: \det A = 0 \iff one of the \lambda_i is zero.

4. The eigenvalues of A^{-1} are 1/\lambda_1, 1/\lambda_2, \ldots If A\mathbf{v} = \lambda\mathbf{v} and A is invertible, multiply both sides by A^{-1}: \mathbf{v} = \lambda A^{-1}\mathbf{v}, so A^{-1}\mathbf{v} = \frac{1}{\lambda}\mathbf{v}.

5. The eigenvalues of A^k are \lambda_1^k, \lambda_2^k, \ldots Apply A to A\mathbf{v} = \lambda\mathbf{v}: A^2\mathbf{v} = A(\lambda\mathbf{v}) = \lambda(A\mathbf{v}) = \lambda^2\mathbf{v}. Repeat for any power k.

The Cayley-Hamilton theorem

Here is a remarkable fact. Take the characteristic polynomial of a matrix — say, for A = \begin{bmatrix} 4 & 2 \\ 1 & 3 \end{bmatrix}, the characteristic polynomial is p(\lambda) = \lambda^2 - 7\lambda + 10.

Now do something that looks illegal: replace \lambda with the matrix A itself:

The claim is that this always equals the zero matrix.

Compute it:

It works.

For a 2 \times 2 matrix with characteristic polynomial \lambda^2 - \text{tr}(A)\,\lambda + \det(A) = 0, the theorem says

For a 3 \times 3 matrix with characteristic polynomial \lambda^3 - c_2\lambda^2 + c_1\lambda - c_0 = 0, the theorem says

Verification for the 3 \times 3 example

Take A = \begin{bmatrix} 2 & 0 & 1 \\ 0 & 3 & 0 \\ 1 & 0 & 2 \end{bmatrix} from Example 2. The characteristic polynomial gives A^3 - 7A^2 + 15A - 9I = O.

Compute A^2 = \begin{bmatrix} 5 & 0 & 4 \\ 0 & 9 & 0 \\ 4 & 0 & 5 \end{bmatrix} and A^3 = \begin{bmatrix} 14 & 0 & 13 \\ 0 & 27 & 0 \\ 13 & 0 & 14 \end{bmatrix}.

Check entry (1,1): 14 - 35 + 30 - 9 = 0. Entry (1,3): 13 - 28 + 15 - 0 = 0. Entry (2,2): 27 - 63 + 45 - 9 = 0. Every entry vanishes — the 3 \times 3 matrix satisfies its own characteristic equation too.

Applications of the Cayley-Hamilton theorem

The Cayley-Hamilton theorem is not just a pretty identity — it has direct computational uses.

Finding A^{-1} from the characteristic equation

For a 2 \times 2 matrix, the Cayley-Hamilton theorem gives A^2 - \text{tr}(A) \cdot A + \det(A) \cdot I = O. If \det A \neq 0, rearrange:

So

For A = \begin{bmatrix} 4 & 2 \\ 1 & 3 \end{bmatrix}: \text{tr}(A) = 7, \det(A) = 10.

Verify: A \cdot A^{-1} = \frac{1}{10}\begin{bmatrix} 4 & 2 \\ 1 & 3 \end{bmatrix}\begin{bmatrix} 3 & -2 \\ -1 & 4 \end{bmatrix} = \frac{1}{10}\begin{bmatrix} 10 & 0 \\ 0 & 10 \end{bmatrix} = I. It works.

Computing higher powers of a matrix

The Cayley-Hamilton theorem lets you express A^n (for any n \geq 2) as a linear combination of I and A (for a 2 \times 2 matrix) or I, A, and A^2 (for 3 \times 3). The idea: use A^2 = \text{tr}(A) \cdot A - \det(A) \cdot I to replace A^2 wherever it appears. Then A^3 = A \cdot A^2 = A \cdot (\text{tr}(A) \cdot A - \det(A) \cdot I) = \text{tr}(A) \cdot A^2 - \det(A) \cdot A, and substitute again for A^2. This reduces any power to a linear combination of I and A.

For A = \begin{bmatrix} 4 & 2 \\ 1 & 3 \end{bmatrix}: A^2 = 7A - 10I.

Without Cayley-Hamilton, computing A^3 requires two full matrix multiplications. With it, you only need scalar arithmetic once you have the relation A^2 = 7A - 10I.

Common confusions

• "Eigenvalues are always real." Not always. A matrix with real entries can have complex eigenvalues. For instance, A = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix} represents a 90° rotation — it has no real eigenvectors (nothing stays on its own line under rotation), and its eigenvalues are \lambda = \pm i. Real symmetric matrices, however, are guaranteed to have all real eigenvalues.

• "Every eigenvalue has exactly one eigenvector." Each eigenvalue has at least one eigenvector, but there can be a whole space of them — a subspace called the eigenspace. Any nonzero vector in that subspace is an eigenvector. In Example 2, \lambda = 3 has a two-dimensional eigenspace.

• "The Cayley-Hamilton theorem says \det(A - AI) = 0." No — you do not substitute A into the determinant formula. You first compute the characteristic polynomial as a polynomial in \lambda, and then substitute A for \lambda in that polynomial. The determinant computation is done with scalars; the substitution into the result is done with matrices.

• "If \lambda = 0 is an eigenvalue, the matrix is useless." Having a zero eigenvalue just means the matrix is singular — it collapses some direction to zero. This is very common and very useful in applications (it tells you the null space of the matrix, which is the eigenspace for \lambda = 0).

Going deeper

Why Cayley-Hamilton is true (the idea)

The full proof of the Cayley-Hamilton theorem requires tools from linear algebra (such as the adjugate of A - \lambda I, whose entries are polynomials in \lambda) and is more involved than a single article can do justice to. But the intuition is accessible.

For each eigenvalue \lambda_i, the characteristic polynomial satisfies p(\lambda_i) = 0 by construction. If A has n linearly independent eigenvectors \mathbf{v}_1, \ldots, \mathbf{v}_n, then for each one:

This uses property 5 (eigenvalues of A^k are \lambda_i^k) and linearity. Since p(A) sends every basis vector to \mathbf{0}, it sends every vector to \mathbf{0} — meaning p(A) = O.

This argument works perfectly when A has n independent eigenvectors (the "diagonalisable" case). For matrices that are not diagonalisable, the proof requires more care — but the result still holds for every square matrix.

Algebraic vs. geometric multiplicity

The algebraic multiplicity of an eigenvalue is its multiplicity as a root of the characteristic polynomial. The geometric multiplicity is the dimension of its eigenspace — the number of linearly independent eigenvectors.

The geometric multiplicity is always at least 1 (every eigenvalue has at least one eigenvector) and at most the algebraic multiplicity. When the two are equal for every eigenvalue, the matrix is diagonalisable. When they differ for some eigenvalue, the matrix has a more complicated structure (described by its Jordan normal form).

In Example 2, \lambda = 3 has algebraic multiplicity 2. Its eigenspace is 2-dimensional (geometric multiplicity = 2), so the multiplicities match and the matrix is diagonalisable.

Why eigenvalues matter

Eigenvalues appear wherever matrices appear, and matrices appear everywhere.

Differential equations. The stability of a system of differential equations \mathbf{x}' = A\mathbf{x} is determined by the eigenvalues of A. If all eigenvalues have negative real part, all solutions decay to zero (the system is stable). If any eigenvalue has positive real part, solutions blow up.

Vibration analysis. The natural frequencies at which a structure vibrates are the square roots of the eigenvalues of a stiffness matrix. When an earthquake hits, the eigenvalues tell you which modes of oscillation the building will experience.

Principal Component Analysis. In data science and machine learning, PCA finds the directions of maximum variance in a dataset by computing the eigenvalues and eigenvectors of the covariance matrix. The largest eigenvalue tells you the direction in which the data varies most.

Where this leads next

Eigenvalues are a gateway to the deeper structure of linear algebra. The most direct continuations:

• Consistency of Systems — the rank conditions that decide whether a system has solutions, and the connection to eigenvalues (\lambda = 0 \iff singular \iff rank < n).
• Properties of Determinants — the computational shortcuts for evaluating determinants, which speed up characteristic equation calculations.
• Special Matrices — symmetric, orthogonal, and Hermitian matrices have particularly nice eigenvalue properties (real eigenvalues, orthogonal eigenvectors).
• Inverse of Matrix — the Cayley-Hamilton route to A^{-1} as an alternative to the adjoint formula.
• Determinants in Geometry — how the determinant (product of eigenvalues) measures area and volume scaling.