📚 Eigenvalues and Eigenvectors in Linear Algebra

• Core Definition and Geometry: Understanding eigenvectors, eigenvalues, and their geometric interpretations as scaling factors.
• Matrix Representation: Connecting linear transformations to $n \times n$ matrices and vector spaces.
• Characteristic Equation & Polynomial: Methods for calculating eigenvalues using determinants and characteristic polynomials.
• Multiplicities and Spectra: Exploring algebraic multiplicity, geometric multiplicity, spectral radius, and eigenspaces.
• Matrix Properties: Key mathematical relationships involving trace, determinant, powers, and matrix inversion.

---

💡 Core Concepts and Definitions

What is an Eigenvalue and Eigenvector?

In linear algebra, an eigenvector (or characteristic vector) is a nonzero vector whose direction remains unchanged (or is precisely reversed) when a given linear transformation is applied to it.

• Eigenvector ($\mathbf{v}$): A vector that is only stretched, shrunk, or reversed in direction by a linear transformation.
• Eigenvalue ($\lambda$): The constant scaling factor (which may be a negative or complex number) by which the eigenvector is multiplied during the transformation.

The fundamental relationship is expressed by the eigenvalue equation: $T(\mathbf{v}) = \lambda \mathbf{v}$

Geometric Interpretation

• Geometrically, vectors are multi-dimensional quantities possessing both magnitude and direction.
• A linear transformation typically rotates, stretches, or shears vectors.
• Eigenvectors undergo no rotation or shear—they are strictly scaled (stretched, shrunk, or inverted).
• The eigenvalue determines the scaling magnitude and direction:
  • If $\lambda > 0$: The vector is stretched ($\lambda > 1$) or shrunk ($0 < \lambda < 1$) without changing direction.
  • If $\lambda < 0$: The vector is scaled and its direction is reversed.
  • If $\lambda = 1$: The vector remains completely unchanged in both length and direction.

---

🗺️ Key Associated Terminology

Term	Definition
Eigensystem	The complete set of all eigenvectors of a linear transformation, each paired with its corresponding eigenvalue.
Eigenspace (Characteristic Space)	The set of all eigenvectors corresponding to a specific eigenvalue, plus the zero vector.
Eigenbasis	An eigenbasis is formed when a set of eigenvectors of a transformation spans the entire domain (i.e., forms a basis).
Spectrum	The complete list of a matrix's eigenvalues, repeated according to their respective multiplicities.
Spectral Radius	The maximum absolute value among all eigenvalues of a given matrix.

---

🔢 Eigenvalues and Eigenvectors of a Matrix

For a finite-dimensional vector space, linear transformations can be explicitly represented as $n \times n$ square matrices.

Given an $n \times n$ matrix $A$ and a nonzero $n$-vector $\mathbf{v}$, if multiplying $A$ by $\mathbf{v}$ yields a scalar multiple of $\mathbf{v}$, the relationship is written as: $A \mathbf{v} = \lambda \mathbf{v}$

The Characteristic Polynomial and Equation

To find eigenvalues computationally, the matrix equation is rearranged: $A \mathbf{v} - \lambda I \mathbf{v} = \mathbf{0} \implies (A - \lambda I)\mathbf{v} = \mathbf{0}$

Key Rule: Equation $(A - \lambda I)\mathbf{v} = \mathbf{0}$ has a nonzero solution $\mathbf{v}$ if and only if the determinant of the matrix $(A - \lambda I)$ is zero: $\det(A - \lambda I) = 0$

• Characteristic Polynomial: Using the Leibniz formula, $\det(A - \lambda I)$ expands into a polynomial of degree $n$ in terms of $\lambda$. Its leading term is always $(-1)^n \lambda^n$.
• Characteristic Equation: Setting the characteristic polynomial equal to zero ($\det(A - \lambda I) = 0$) allows you to solve for the eigenvalues $\lambda$.

Example: A $2 \times 2$ Matrix

Consider the matrix: $A = \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}$

1. Compute the characteristic polynomial using $\det(A - \lambda I) = 0$: $\det \begin{vmatrix} 2 - \lambda & 1 \\ 1 & 2 - \lambda \end{vmatrix} = (2 - \lambda)^2 - 1 = 3 - 4\lambda + \lambda^2 = 0$
2. Solve for eigenvalues by factoring: $(\lambda - 1)(\lambda - 3) = 0 \implies \lambda_1 = 1, \quad \lambda_2 = 3$
3. Find corresponding eigenvectors by solving $(A - \lambda I)\mathbf{v} = 0$:
   • For $\lambda = 1$: $\mathbf{v}_{\lambda=1} = \begin{bmatrix} 1 \\ -1 \end{bmatrix}$
   • For $\lambda = 3$: $\mathbf{v}_{\lambda=3} = \begin{bmatrix} 1 \\ 1 \end{bmatrix}$

---

🔀 Multiplicities of Eigenvalues

1. Algebraic Multiplicity ($\mu_A$)

• Definition: The multiplicity of an eigenvalue as a root of the characteristic polynomial.
• Constraint: If an eigenvalue $\lambda_i$ appears $k$ times as a root, its algebraic multiplicity is $\mu_A(\lambda_i) = k$.
• The sum of all algebraic multiplicities equals the matrix dimension $n$: $\sum_{i=1}^{d} \mu_A(\lambda_i) = n$

2. Geometric Multiplicity ($\gamma_A$)

• Definition: The dimension of the eigenspace associated with $\lambda$, which equals the maximum number of linearly independent eigenvectors for that eigenvalue.
• Formula: $\gamma_A(\lambda) = n - \text{rank}(A - \lambda I)$

Fundamental Inequality of Multiplicities

For any eigenvalue $\lambda$ of a matrix $A$, the following inequalities always hold: $1 \le \gamma_A(\lambda) \le \mu_A(\lambda) \le n$

---

📋 Additional Mathematical Properties of Matrix Eigenvalues

Let $A$ be an $n \times n$ complex matrix with eigenvalues $\lambda_1, \lambda_2, \dots, \lambda_n$ (counted with algebraic multiplicities). The eigenvalues govern several critical matrix operations:

Property	Mathematical Expression	Description
Trace	$\operatorname{tr}(A) = \sum_{i=1}^{n} a_{ii} = \sum_{i=1}^{n} \lambda_i$	The sum of the diagonal elements equals the sum of all eigenvalues.
Determinant	$\det(A) = \prod_{i=1}^{n} \lambda_i$	The determinant of a matrix equals the product of all its eigenvalues.
Matrix Powers	$A^k \implies \lambda_1^k, \lambda_2^k, \dots, \lambda_n^k$	Raising a matrix to a power $k$ raises all its eigenvalues to the same power.
Scalar Shift	$A + \alpha I \implies \lambda_1 + \alpha, \dots, \lambda_n + \alpha$	Adding a scalar multiple of the identity matrix shifts all eigenvalues by that scalar.
Polynomials	$P(A) \implies P(\lambda_1), \dots, P(\lambda_n)$	Applying a polynomial function $P$ to a matrix applies it directly to its eigenvalues.
Invertibility	$\lambda_i \neq 0$ for all $i$	$A$ is invertible if and only if every eigenvalue is nonzero.
Inverse Matrix	$A^{-1} \implies \frac{1}{\lambda_1}, \dots, \frac{1}{\lambda_n}$	If $A$ is invertible, the eigenvalues of its inverse are the reciprocals of its original eigenvalues.