Eigenvectors are determined by solving the characteristic equation of a matrix, finding eigenvalues, and then solving a system of linear equations for each eigenvalue.
Delving into linear algebra, we encounter concepts that illuminate how transformations reshape vectors. Eigenvectors are fundamental to understanding these transformations, revealing the directions that remain unchanged, only scaled, by a linear mapping. This exploration will guide you through the precise steps to uncover these special vectors.
Understanding Eigenvalues and Eigenvectors
At its heart, an eigenvector of a linear transformation is a non-zero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding scalar factor is known as the eigenvalue. Together, they form an eigenpair, which reveals intrinsic properties of the matrix representing the transformation.
Consider a matrix $A$ and a vector $\mathbf{v}$. If $A\mathbf{v} = \lambda\mathbf{v}$, where $\lambda$ is a scalar, then $\mathbf{v}$ is an eigenvector of $A$, and $\lambda$ is its corresponding eigenvalue. This relationship means applying the matrix $A$ to $\mathbf{v}$ simply stretches or shrinks $\mathbf{v}$ without changing its direction. This property is incredibly useful across many scientific and engineering disciplines.
The Geometric Interpretation
Geometrically, an eigenvector points in a direction that is invariant under the matrix transformation. While the vector itself might be scaled (stretched or compressed), its orientation in space remains fixed. For example, a rotation matrix applied to an eigenvector might only scale it if the eigenvector lies along the axis of rotation, assuming the rotation is in 3D space.
In two dimensions, if you have a transformation that stretches space along one axis and compresses it along another, the eigenvectors would align with these stretching and compressing axes. The eigenvalues would then represent the scaling factors along those specific directions.
The Characteristic Equation: Finding Eigenvalues
Before we can calculate eigenvectors, we must first find their associated eigenvalues. The eigenvalues are the roots of the characteristic equation, which arises directly from the definition of an eigenpair.
Starting with $A\mathbf{v} = \lambda\mathbf{v}$, we can rearrange this equation. Since $\mathbf{v}$ is non-zero, we cannot simply divide by it. Instead, we introduce the identity matrix $I$ to maintain matrix operations:
- Rewrite the equation: $A\mathbf{v} – \lambda\mathbf{v} = \mathbf{0}$
- Factor out $\mathbf{v}$: $(A – \lambda I)\mathbf{v} = \mathbf{0}$
For a non-zero vector $\mathbf{v}$ to satisfy this equation, the matrix $(A – \lambda I)$ must be singular, meaning its determinant must be zero. This condition provides the characteristic equation:
$\det(A – \lambda I) = 0$
Steps to Determine Eigenvalues
To find the eigenvalues, follow these steps:
- Form the matrix $(A – \lambda I)$ by subtracting $\lambda$ from each diagonal entry of matrix $A$.
- Calculate the determinant of this new matrix.
- Set the determinant equal to zero. This polynomial equation in $\lambda$ is the characteristic equation.
- Solve the characteristic equation for $\lambda$. The solutions are the eigenvalues. A matrix of size $n \times n$ will have $n$ eigenvalues, counting multiplicity.
For a $2 \times 2$ matrix $A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$, the characteristic equation is $(a-\lambda)(d-\lambda) – bc = 0$, which simplifies to $\lambda^2 – (a+d)\lambda + (ad-bc) = 0$. Here, $(a+d)$ is the trace of $A$, and $(ad-bc)$ is the determinant of $A$.
How To Calculate Eigenvectors: The Core Process
Once you have determined the eigenvalues, the next step is to calculate the corresponding eigenvectors. Each eigenvalue will have at least one associated eigenvector. The process involves substituting each eigenvalue back into the equation $(A – \lambda I)\mathbf{v} = \mathbf{0}$ and solving for $\mathbf{v}$.
This equation represents a homogeneous system of linear equations. Since $(A – \lambda I)$ is singular for an eigenvalue $\lambda$, there will be non-trivial solutions for $\mathbf{v}$. These non-trivial solutions form the eigenspace associated with $\lambda$, and any non-zero vector in this eigenspace is an eigenvector.
Detailed Steps for Eigenvector Calculation
- Select an Eigenvalue: Choose one of the eigenvalues, say $\lambda_1$, that you found from the characteristic equation.
- Form the System: Substitute $\lambda_1$ into the matrix expression $(A – \lambda_1 I)$.
- Set up the Homogeneous System: Construct the system of linear equations $(A – \lambda_1 I)\mathbf{v} = \mathbf{0}$. If $\mathbf{v} = \begin{pmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{pmatrix}$, then this becomes a system of equations in terms of $x_1, x_2, \dots, x_n$.
- Solve the System: Use methods like Gaussian elimination or row reduction to solve this system. You will find that there is at least one free variable, leading to an infinite number of solutions.
- Express the Eigenvector: Write the solution in parametric form. The non-zero vector(s) you derive represent the eigenvector(s) for $\lambda_1$. Often, we choose a simple non-zero value for the free variable to get a representative eigenvector.
- Repeat for All Eigenvalues: Perform steps 1-5 for each distinct eigenvalue.
It’s important to remember that if $\mathbf{v}$ is an eigenvector, then any non-zero scalar multiple $c\mathbf{v}$ is also an eigenvector for the same eigenvalue. This is because $A(c\mathbf{v}) = c(A\mathbf{v}) = c(\lambda\mathbf{v}) = \lambda(c\mathbf{v})$.
| Eigenvalue Calculation | Eigenvector Calculation |
|---|---|
| Form $(A – \lambda I)$ matrix. | Substitute $\lambda$ into $(A – \lambda I)$. |
| Calculate $\det(A – \lambda I)$. | Form the system $(A – \lambda I)\mathbf{v} = \mathbf{0}$. |
| Solve $\det(A – \lambda I) = 0$ for $\lambda$. | Solve the homogeneous system for $\mathbf{v}$. |
Handling Repeated Eigenvalues
Sometimes, the characteristic equation yields eigenvalues with a multiplicity greater than one. This means an eigenvalue appears multiple times as a root of the characteristic polynomial. For instance, if $\lambda=2$ is a root of multiplicity two, it’s a repeated eigenvalue.
When an eigenvalue $\lambda$ has an algebraic multiplicity $m$ (meaning it appears $m$ times as a root), it can have anywhere from 1 to $m$ linearly independent eigenvectors. The number of linearly independent eigenvectors associated with $\lambda$ is called its geometric multiplicity. The geometric multiplicity is always less than or equal to the algebraic multiplicity.
Finding Eigenvectors for Repeated Eigenvalues
The process for finding eigenvectors for repeated eigenvalues remains the same: substitute the repeated eigenvalue into $(A – \lambda I)\mathbf{v} = \mathbf{0}$ and solve the system. However, the solution space might be higher-dimensional.
- If the geometric multiplicity equals the algebraic multiplicity, you will find $m$ linearly independent eigenvectors.
- If the geometric multiplicity is less than the algebraic multiplicity, you will find fewer than $m$ linearly independent eigenvectors. In such cases, the matrix is called “defective,” and additional concepts like generalized eigenvectors are sometimes introduced in more advanced studies. For basic eigenvector calculation, we simply find all linearly independent eigenvectors available from the homogeneous system.
When solving the system for a repeated eigenvalue, you might find two or more free variables, which directly indicates a higher geometric multiplicity and thus multiple linearly independent eigenvectors for that single eigenvalue.
Orthogonality and Normalization
Eigenvectors possess properties that are often useful, especially in specific contexts like symmetric matrices. For real symmetric matrices, eigenvectors corresponding to distinct eigenvalues are always orthogonal. This means their dot product is zero.
If $A$ is a symmetric matrix, and $\mathbf{v}_1$ and $\mathbf{v}_2$ are eigenvectors corresponding to distinct eigenvalues $\lambda_1$ and $\lambda_2$ respectively, then $\mathbf{v}_1 \cdot \mathbf{v}_2 = 0$. This property simplifies many analyses, particularly in principal component analysis (PCA).
Normalizing Eigenvectors
While any non-zero scalar multiple of an eigenvector is still an eigenvector, it is often useful to normalize eigenvectors. Normalization means scaling the eigenvector so its length (or Euclidean norm) is 1. This is done by dividing each component of the eigenvector by its magnitude.
If $\mathbf{v} = \begin{pmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{pmatrix}$, its magnitude is $||\mathbf{v}|| = \sqrt{x_1^2 + x_2^2 + \dots + x_n^2}$. The normalized eigenvector $\hat{\mathbf{v}}$ is then $\hat{\mathbf{v}} = \frac{1}{||\mathbf{v}||}\mathbf{v}$.
Normalized eigenvectors are particularly useful when comparing directions or when forming orthonormal bases, as they provide a consistent representation without arbitrary scaling factors.
| Property | Description |
|---|---|
| Direction Invariance | Eigenvectors maintain their direction (or directly opposite) under the matrix transformation, only scaled. |
| Non-Zero | By definition, eigenvectors are always non-zero vectors. |
| Scalar Multiples | Any non-zero scalar multiple of an eigenvector is also an eigenvector for the same eigenvalue. |
| Orthogonality (Symmetric Matrices) | Eigenvectors corresponding to distinct eigenvalues of a real symmetric matrix are orthogonal. |
Applications of Eigenvectors
Eigenvectors and eigenvalues are not merely theoretical constructs; they are powerful tools with broad applications across various fields. Their ability to reveal the fundamental modes of behavior or principal directions within complex systems makes them indispensable.
Engineering and Physics
- Vibration Analysis: In structural engineering, eigenvectors represent the natural modes of vibration of a structure (e.g., a bridge or building), while eigenvalues correspond to the frequencies of these vibrations. Understanding these helps in designing structures that resist resonance.
- Quantum Mechanics: In quantum mechanics, eigenvectors represent the stationary states of a system, and eigenvalues correspond to the measurable energies or other observable quantities.
- Stability Analysis: Eigenvalues determine the stability of dynamic systems, such as control systems or ecological models. If eigenvalues have positive real parts, the system tends to be unstable.
Data Science and Machine Learning
- Principal Component Analysis (PCA): PCA is a dimensionality reduction technique that uses eigenvectors to find the principal components of a dataset. These components are orthogonal directions that capture the most variance in the data, effectively reducing noise and complexity while retaining important information.
- Image Compression: Eigenvectors are used in image processing for compression by identifying the most significant features (eigenfaces in facial recognition, for example) that contribute most to the image’s information.
- Graph Theory: Eigenvalues and eigenvectors of adjacency matrices are used in spectral graph theory to analyze network structures, identify communities, and rank nodes (e.g., Google’s PageRank algorithm used a form of eigenvector centrality).