Vector Spaces: Linear Algebra Study Notes

📐 Vector Spaces

Main Topics Covered

• What a vector space is: sets of vectors that can be added and scaled, over a field of scalars
• Definition, the vector axioms, and basic consequences
• Linear combinations, independence, subspaces, span, bases, dimension and coordinates
• Standard examples: arrows, ordered pairs, coordinate spaces, complex numbers, function spaces, linear equations
• Linear maps, isomorphisms, matrices, determinants, eigenvalues and eigenvectors
• Constructions: subspaces, quotient spaces, kernels and images, direct products and sums, tensor products
• Extra structure: order, norms, inner products, topology, Banach and Hilbert spaces, algebras
• Related structures: vector bundles, modules, affine and projective spaces

---

💡 Core Idea

A vector space (also called a linear space) is a set whose elements, often called vectors, can be added together and multiplied ("scaled") by numbers called scalars.

• The operations of vector addition and scalar multiplication must satisfy requirements called the vector axioms.
• Real vector spaces and complex vector spaces use real numbers and complex numbers as scalars.
• More generally, scalars can be elements of any field.
• Vector spaces generalize Euclidean vectors, which model physical quantities such as forces and velocity that have both a magnitude and a direction.
• They are fundamental to linear algebra, together with matrices, which allow computing in vector spaces. This gives a concise way to manipulate and study systems of linear equations.

Dimension

Vector spaces are characterized by their dimension, which roughly specifies the number of independent directions in the space.

• Two vector spaces over the same field with the same dimension are isomorphic: properties depending only on the vector-space structure are exactly the same.
• Finite-dimensional: the dimension is a natural number. These occur naturally in geometry and related areas.
• Infinite-dimensional: the dimension is an infinite cardinal. These occur in many areas of mathematics. For example, polynomial rings are countably infinite-dimensional, and many function spaces have the cardinality of the continuum as a dimension.

Vector spaces with extra structure

Kind of extra structure	Examples
Algebras	Field extensions, polynomial rings, associative algebras, Lie algebras
Topological vector spaces	Function spaces, inner product spaces, normed spaces, Hilbert spaces, Banach spaces

---

📖 Definition and Basic Properties

In this topic, vectors are written in boldface to distinguish them from scalars.

A vector space over a field $F$ is a non-empty set $V$ together with a binary operation and a binary function that satisfy the eight axioms. The elements of $V$ are called vectors; the elements of $F$ are called scalars.

• Vector addition (or simply addition) assigns to any two vectors $\mathbf{v}$ and $\mathbf{w}$ in $V$ a third vector in $V$, written $\mathbf{v}+\mathbf{w}$, called their sum.
• Scalar multiplication assigns to any scalar $a \in F$ and any vector $\mathbf{v} \in V$ another vector in $V$, denoted $a\mathbf{v}$.
• The eight axioms must hold for every $\mathbf{u}, \mathbf{v}, \mathbf{w}$ in $V$ and $a, b$ in $F$.

The eight axioms

For all $\mathbf{u},\mathbf{v},\mathbf{w} \in V$ and $a,b \in F$:

Axiom	Statement
Associativity of addition	$\mathbf{u}+(\mathbf{v}+\mathbf{w})=(\mathbf{u}+\mathbf{v})+\mathbf{w}$
Commutativity of addition	$\mathbf{u}+\mathbf{v}=\mathbf{v}+\mathbf{u}$
Identity element of addition	There is $\mathbf{0}\in V$ with $\mathbf{v}+\mathbf{0}=\mathbf{v}$ for all $\mathbf{v}$
Inverse elements of addition	For every $\mathbf{v}$ there is $-\mathbf{v}\in V$ with $\mathbf{v}+(-\mathbf{v})=\mathbf{0}$
Compatibility of scalar multiplication with field multiplication	$a(b\mathbf{v})=(ab)\mathbf{v}$
Identity element of scalar multiplication	$1\mathbf{v}=\mathbf{v}$, where $1$ is the multiplicative identity of $F$
Distributivity over vector addition	$a(\mathbf{u}+\mathbf{v})=a\mathbf{u}+a\mathbf{v}$
Distributivity over field addition	$(a+b)\mathbf{v}=a\mathbf{v}+b\mathbf{v}$

Naming by scalar field

Scalar field	Name
Real numbers	Real vector space
Complex numbers	Complex vector space
Arbitrary field $F$	$F$-vector space, or vector space over $F$

Real and complex are the most common cases, but arbitrary fields are also commonly considered.

An equivalent, more concise definition

This version is less elementary:

• The first four axioms (about vector addition) say that a vector space is an abelian group under addition.
• The four remaining axioms (about scalar multiplication) say that this operation defines a ring homomorphism from the field $F$ into the endomorphism ring of this group.
  • Distributivity of scalar multiplication over vector addition means multiplication by a scalar $a$ is an endomorphism of the group.
  • The remaining three axioms establish that the function mapping a scalar $a$ to multiplication by $a$ is a ring homomorphism from the field to the endomorphism ring.
• Even more concisely: a vector space is a module over a field.

Subtraction and direct consequences of the axioms

Subtraction is defined as

$\mathbf{v}-\mathbf{w}=\mathbf{v}+(-\mathbf{w}).$

For every $s \in F$ and $\mathbf{v} \in V$:

• $0\mathbf{v}=\mathbf{0}$
• $s\mathbf{0}=\mathbf{0}$
• $(-1)\mathbf{v}=-\mathbf{v}$
• $s\mathbf{v}=\mathbf{0}$ implies $s=0$ or $\mathbf{v}=\mathbf{0}$

---

🧱 Bases, Coordinates, and Subspaces

Key definitions

Concept	Definition
Linear combination	For a set $G$ in an $F$-vector space $V$: an element of the form $a_1\mathbf{g}_1+a_2\mathbf{g}_2+\cdots+a_k\mathbf{g}_k$ with $a_i \in F$ and $\mathbf{g}_i \in G$. The $a_i$ are the coefficients.
Linear independence	Elements of $G$ are independent if no element can be written as a linear combination of the others.
Linear subspace	A non-empty subset $W$ of $V$ closed under vector addition and scalar multiplication.
Linear span	The smallest linear subspace containing $G$ (the intersection of all subspaces containing $G$); equivalently, the set of all linear combinations of elements of $G$.
Basis	A subset whose elements are linearly independent and span the space.
Dimension	The common cardinality of all bases of the space.

Equivalent forms of linear independence

• No element of $G$ is a linear combination of the others.
• Two linear combinations of elements of $G$ define the same element of $V$ if and only if they have the same coefficients.
• A linear combination gives the zero vector if and only if all its coefficients are zero.

Linear subspaces

• Closure means the sum of two elements of $W$ and the product of an element of $W$ by a scalar both belong to $W$.
• Hence every linear combination of elements of $W$ belongs to $W$.
• A subspace is itself a vector space under the induced operations: closure implies the axioms are satisfied.
• Every intersection of linear subspaces is a linear subspace.

Span

• If $W$ is the span of $G$, then $G$ spans or generates $W$, and $G$ is a spanning set or generating set of $W$.

Basis and dimension

• Every vector space has at least one basis, and in general many.
• All bases of a vector space have the same cardinality, called the dimension. This is a fundamental property.
• Bases are a fundamental tool, especially in finite dimension.
• In the infinite-dimensional case, the existence of infinite bases (often called Hamel bases) depends on the axiom of choice. So in general no basis can be explicitly described.
  • Example: the real numbers form an infinite-dimensional vector space over the rational numbers, for which no specific basis is known.

Coordinates

Take a basis $(\mathbf{b}_1,\mathbf{b}_2,\ldots,\mathbf{b}_n)$ of a vector space $V$ of dimension $n$ over $F$. Every $\mathbf{v} \in V$ can be written

$\mathbf{v}=a_1\mathbf{b}_1+\cdots+a_n\mathbf{b}_n,$

with $a_1,\dots,a_n \in F$, and this decomposition is unique.

• The scalars $a_1,\ldots,a_n$ are the coordinates of $\mathbf{v}$ on the basis (also the coefficients of the decomposition).
• The $n$-tuple of coordinates is the coordinate vector of $\mathbf{v}$. The set $F^n$ of $n$-tuples is a vector space under componentwise operations, with dimension $n$.
• The one-to-one correspondence between vectors and coordinate vectors maps addition to addition and scalar multiplication to scalar multiplication. It is a vector space isomorphism, so reasoning and computation on vectors can be translated into reasoning and computation on coordinates.

---

🧪 Examples

Arrows in the plane

• Arrows in a fixed plane, all starting at one fixed point; used in physics for forces and velocities.
• Sum: for arrows $\mathbf{v}$ and $\mathbf{w}$, the parallelogram they span has a diagonal arrow starting at the origin. This is $\mathbf{v}+\mathbf{w}$.
  • For two arrows on the same line, the sum has length equal to the sum or difference of the lengths, depending on whether the directions agree.
• Scaling: for a positive real number $a$, $a\mathbf{v}$ has the same direction as $\mathbf{v}$ but its length multiplied by $a$. If $a$ is negative, $a\mathbf{v}$ points in the opposite direction.
• Example: with $a=2$, $2\mathbf{w}$ has the same direction as $\mathbf{w}$ and double the length; equivalently $2\mathbf{w}=\mathbf{w}+\mathbf{w}$. Also $(-1)\mathbf{v}=-\mathbf{v}$ has the opposite direction and the same length.

Ordered pairs of numbers

Pairs $(x,y)$ of real numbers, where order matters:

$\begin{aligned}(x_1,y_1)+(x_2,y_2)&=(x_1+x_2,\,y_1+y_2),\\ a(x,y)&=(ax,\,ay).\end{aligned}$

The arrow example reduces to this one if an arrow is represented by the Cartesian coordinates of its endpoint.

Coordinate space

• The simplest example over a field $F$ is $F$ itself, with its addition as vector addition and its multiplication as scalar multiplication.
• More generally, all $n$-tuples $(a_1,a_2,\dots,a_n)$ of elements of $F$ form a vector space $F^n$, the coordinate space.
• $n=1$ gives the simplest example. $F=\mathbf{R}$, $n=2$ gives the ordered pairs above.

Complex numbers and other field extensions

• The complex numbers $\mathbf{C}$ (numbers $x+iy$ with $x,y$ real) form a vector space over the reals: $(x+iy)+(a+ib)=(x+a)+i(y+b)$ and $c\cdot(x+iy)=(c\cdot x)+i(c\cdot y)$.
• The axioms follow from the rules of complex arithmetic. This space is isomorphic to the space of ordered pairs of real numbers: $x+iy$ corresponds to $(x,y)$ in the complex plane.
• More generally, a field $F$ containing a smaller field $E$ is an $E$-vector space. For example, $\mathbf{C}$ is a vector space over $\mathbf{R}$, and the field extension $\mathbf{Q}(i\sqrt{5})$ is a vector space over $\mathbf{Q}$.

Function spaces

• Functions from any fixed set $\Omega$ to a field $F$ form a vector space under pointwise operations: $(f+g)(w)=f(w)+g(w)$, and similarly for scalar multiplication.
• Properties such as continuity, integrability or differentiability are well-behaved under linearity: sums and scalar multiples keep the property. So such functions form vector spaces, studied in functional analysis.

Linear equations

Solutions of systems of homogeneous linear equations form vector spaces. For example, the solutions of

$\begin{aligned}a+3b+c&=0\\ 4a+2b+2c&=0\end{aligned}$

are the triples with arbitrary $a$, $b=a/2$, and $c=-5a/2$. Sums and scalar multiples of such triples still satisfy the same ratios, so they are solutions too.

• Matrices condense the equations into one vector equation $A\mathbf{x}=\mathbf{0}$, where
  • $A=\begin{bmatrix}1&3&1\\4&2&2\end{bmatrix}$ holds the coefficients,
  • $\mathbf{x}=(a,b,c)$,
  • $A\mathbf{x}$ is the matrix product, and $\mathbf{0}=(0,0)$ is the zero vector.
• Solutions of homogeneous linear differential equations also form vector spaces. For example, one such equation yields $f(x)=ae^{-x}+bxe^{-x}$, where $a,b$ are arbitrary constants and $e^x$ is the natural exponential function.

---

🔁 Linear Maps and Matrices

A linear map (linear transformation) between vector spaces preserves sums and scalar multiplication:

$\begin{aligned}f(\mathbf{v}+\mathbf{w})&=f(\mathbf{v})+f(\mathbf{w}),\\ f(a\cdot\mathbf{v})&=a\cdot f(\mathbf{v})\end{aligned}$

for all $\mathbf{v},\mathbf{w} \in V$ and all $a \in F$.

Isomorphisms

• An isomorphism is a linear map $f:V\to W$ with an inverse map $g:W\to V$ such that $f\circ g$ and $g\circ f$ are identity maps. Equivalently, $f$ is both one-to-one (injective) and onto (surjective).
• If an isomorphism exists, $V$ and $W$ are isomorphic: essentially identical as vector spaces, since identities in $V$ are transported to $W$ via $f$ and back via $g$.
• Example: arrows in the plane and ordered pairs of numbers are isomorphic. A planar arrow from the origin is expressed as a pair via its $x$- and $y$-components; conversely a pair $(x,y)$ gives the arrow going $x$ right (left if negative) and $y$ up (down if negative).

Spaces of linear maps and duals

• Linear maps $V\to W$ form a vector space $\mathrm{Hom}_F(V,W)$, also written $L(V,W)$.
• The space of linear maps from $V$ to $F$ is the dual vector space $V^*$.
• The injective natural map $V\to V^{**}$ embeds any vector space into its bidual; it is an isomorphism if and only if the space is finite-dimensional.

Classification by dimension

• Once a basis of $V$ is chosen, a linear map $f:V\to W$ is completely determined by the images of the basis vectors.
• If $\dim V=\dim W$, a 1-to-1 correspondence between fixed bases gives a linear map sending each basis element of $V$ to the corresponding one of $W$. It is an isomorphism by its very definition.
• Therefore two vector spaces over a given field are isomorphic if and only if their dimensions agree. Any vector space over a field is classified up to isomorphism by one number, its dimension.
• In particular, any $n$-dimensional $F$-vector space $V$ is isomorphic to $F^n$. There is no canonical isomorphism: an isomorphism $\varphi:F^n\to V$ is equivalent to a choice of basis of $V$ (mapping the standard basis of $F^n$ to $V$).

Matrices

• Matrices encode linear maps as rectangular arrays of scalars.
• Any $m$-by-$n$ matrix $A$ gives a linear map from $F^n$ to $F^m$:

$\mathbf{x}=(x_1,x_2,\ldots,x_n)\mapsto\left(\sum_{j=1}^{n}a_{1j}x_j,\ \sum_{j=1}^{n}a_{2j}x_j,\ \ldots,\ \sum_{j=1}^{n}a_{mj}x_j\right),$

which is also matrix multiplication of $A$ with the coordinate vector $\mathbf{x}$.

• After choosing bases of $V$ and $W$, any linear map $f:V\to W$ is uniquely represented by a matrix.
• The determinant $\det(A)$ of a square matrix is a scalar telling whether the map is an isomorphism: it is one if and only if the determinant is nonzero.
• The linear transformation of $\mathbf{R}^n$ given by a real $n$-by-$n$ matrix is orientation preserving if and only if its determinant is positive.

Eigenvalues and eigenvectors

• Endomorphisms are linear maps $f:V\to V$; vectors can be compared with their images $f(\mathbf{v})$.
• A nonzero $\mathbf{v}$ with $\lambda\mathbf{v}=f(\mathbf{v})$ ($\lambda$ a scalar) is an eigenvector of $f$ with eigenvalue $\lambda$. Equivalently, $\mathbf{v}$ is in the kernel of $f-\lambda\cdot\mathrm{Id}$.
• If $V$ is finite-dimensional, $f$ having eigenvalue $\lambda$ is equivalent to

$\det(f-\lambda\cdot\operatorname{Id})=0.$

• The left side is a polynomial in $\lambda$, the characteristic polynomial of $f$.
• If the field is large enough to contain a zero of this polynomial (automatic for algebraically closed fields such as $\mathbf{C}$), any linear map has at least one eigenvector.
• $V$ may or may not have an eigenbasis (a basis of eigenvectors); this is governed by the Jordan canonical form.
• The set of all eigenvectors for one eigenvalue forms a vector space, the eigenspace.

---

🏗️ Basic Constructions

Subspaces and quotient spaces

• A nonempty subset $W\subseteq V$ closed under addition and scalar multiplication (so it contains $\mathbf{0}$) is a linear subspace. It is a vector space over the same field.
• The span of a set $S$ is the intersection of all subspaces containing $S$: the smallest such subspace, consisting of all linear combinations of elements of $S$.
• Subspaces of dimension 1 and 2 are called a line (vector line) and a plane. In an $n$-dimensional space, a subspace of dimension $n-1$ is a hyperplane.

Quotient space $V/W$ ("$V$ modulo $W$") for a subspace $W\subseteq V$:

• As a set it consists of $\mathbf{v}+W=\{\mathbf{v}+\mathbf{w}:\mathbf{w}\in W\}$ for arbitrary $\mathbf{v}\in V$.
• Sum: $(\mathbf{v}_1+W)+(\mathbf{v}_2+W)=(\mathbf{v}_1+\mathbf{v}_2)+W$. Scalar multiplication: $a\cdot(\mathbf{v}+W)=(a\cdot\mathbf{v})+W$.
• Key point: $\mathbf{v}_1+W=\mathbf{v}_2+W$ if and only if $\mathbf{v}_1-\mathbf{v}_2\in W$. The quotient "forgets" information contained in $W$.

Kernel and image

• The kernel $\ker(f)$ of $f:V\to W$ is the set of vectors mapped to $\mathbf{0}$. The kernel and the image $\operatorname{im}(f)=\{f(\mathbf{v}):\mathbf{v}\in V\}$ are subspaces of $V$ and $W$ respectively.
• For $\mathbf{x}\mapsto A\mathbf{x}$, the kernel is the set of solutions of $A\mathbf{x}=\mathbf{0}$, i.e. the solutions of the homogeneous system belonging to $A$.
• This extends to linear differential equations $a_0f+a_1\frac{df}{dx}+\cdots+a_n\frac{d^nf}{dx^n}=0$, where the $a_i$ may be functions of $x$. The map

$f\mapsto D(f)=\sum_{i=0}^{n}a_i\frac{d^if}{dx^i}$

is linear (derivatives appear linearly), called a linear differential operator. Differentiation is linear: $(f+g)'=f'+g'$ and $(c\cdot f)'=c\cdot f'$. So the solutions of $D(f)=0$ form a vector space (over $\mathbf{R}$ or $\mathbf{C}$).

• Because kernels and images exist, vector spaces over a fixed field form an abelian category, behaving much like abelian groups. So the first isomorphism theorem (the rank-nullity theorem in matrix terms)

$V/\ker(f)\;\equiv\;\operatorname{im}(f)$

and the second and third isomorphism theorems can be formulated and proven much as for groups.

Direct product and direct sum

Two ways to combine an indexed family of vector spaces $V_i$, $i\in I$:

Construction	Elements	Operations
Direct product	All tuples $(\mathbf{v}_i)_{i\in I}$ with $\mathbf{v}_i\in V_i$	Componentwise
Direct sum (coproduct)	Only tuples with finitely many nonzero vectors	Componentwise

If the index set $I$ is finite the two agree; in general they differ.

Tensor product

The tensor product $V\otimes_F W$ (or $V\otimes W$) is a central notion of multilinear algebra, which extends notions like linear maps to several variables.

• A map $g:V\times W\to X$ is bilinear if it is linear in each variable when the other is fixed.
• $V\otimes W$ is the vector space of finite formal sums of symbols called tensors

$\mathbf{v}_1\otimes\mathbf{w}_1+\mathbf{v}_2\otimes\mathbf{w}_2+\cdots+\mathbf{v}_n\otimes\mathbf{w}_n,$

subject to the rules

$\begin{aligned}a\cdot(\mathbf{v}\otimes\mathbf{w})&=(a\cdot\mathbf{v})\otimes\mathbf{w}=\mathbf{v}\otimes(a\cdot\mathbf{w}),\quad a\text{ a scalar}\\(\mathbf{v}_1+\mathbf{v}_2)\otimes\mathbf{w}&=\mathbf{v}_1\otimes\mathbf{w}+\mathbf{v}_2\otimes\mathbf{w}\\\mathbf{v}\otimes(\mathbf{w}_1+\mathbf{w}_2)&=\mathbf{v}\otimes\mathbf{w}_1+\mathbf{v}\otimes\mathbf{w}_2.\end{aligned}$

• These rules make the map $f:V\times W\to V\otimes W$, $(\mathbf{v},\mathbf{w})\mapsto\mathbf{v}\otimes\mathbf{w}$, bilinear.
• Universal property: for any vector space $X$ and bilinear $g:V\times W\to X$, there is a unique map $u$ with $u\circ f=g$, i.e. $u(\mathbf{v}\otimes\mathbf{w})=g(\mathbf{v},\mathbf{w})$. This is an instance of defining objects indirectly by specifying maps from or to them, a method much used in advanced abstract algebra.

---

🧭 Vector Spaces with Additional Structure

Linear algebra fully classifies vector spaces by dimension, but plain vector spaces cannot address whether a sequence of functions converges, and addition allows only finitely many terms in a sum. The needs of functional analysis therefore require extra structure.

Order

• A vector space may carry a partial order $\leq$, so some vectors can be compared. For example $\mathbf{R}^n$ can be ordered componentwise.
• Ordered vector spaces (for example Riesz spaces) are fundamental to Lebesgue integration, which expresses a function as a difference of two positive functions: $f=f^{+}-f^{-}$, with $f^{+}$ the positive part and $f^{-}$ the negative part.

Normed vector spaces and inner product spaces

• A norm measures lengths of vectors; an inner product measures angles between vectors. They are written $|\mathbf{v}|$ and $\langle\mathbf{v},\mathbf{w}\rangle$.
• An inner product also defines lengths through the associated norm.
• Coordinate space $F^n$ has the standard dot product:

$\langle\mathbf{x},\mathbf{y}\rangle=\mathbf{x}\cdot\mathbf{y}=x_1y_1+\cdots+x_ny_n.$

• In $\mathbf{R}^2$ this matches the usual angle between vectors, by the law of cosines:

$\mathbf{x}\cdot\mathbf{y}=\cos\left(\angle(\mathbf{x},\mathbf{y})\right)\cdot|\mathbf{x}|\cdot|\mathbf{y}|.$

• Vectors with $\langle\mathbf{x},\mathbf{y}\rangle=0$ are orthogonal.
• Minkowski space: $\mathbf{R}^4$ with the Lorentz product

$\langle\mathbf{x}|\mathbf{y}\rangle=x_1y_1+x_2y_2+x_3y_3-x_4y_4.$

It is not positive definite: $\langle\mathbf{x}|\mathbf{x}\rangle$ can be negative, e.g. for $\mathbf{x}=(0,0,0,1)$. The fourth coordinate corresponds to time, making it useful for special relativity. In other conventions time is the first ("zeroeth") component, giving $\langle\mathbf{x}|\mathbf{y}\rangle=-x_0y_0+x_1y_1+x_2y_2+x_3y_3$.

Topological vector spaces

Convergence is handled by giving $V$ a compatible topology: addition and scalar multiplication must be continuous maps. Roughly, if $\mathbf{x},\mathbf{y}$ and $a$ vary by a bounded amount, so do $\mathbf{x}+\mathbf{y}$ and $a\mathbf{x}$. The field $F$ must also carry a topology (commonly the reals or complex numbers).

• Series: $\sum_{i=1}^{\infty}f_i=\lim_{n\to\infty}\left(f_1+\cdots+f_n\right)$, the limit of partial sums. In function spaces the mode of convergence depends on the topology; pointwise and uniform convergence are two prominent examples.
• Complete: every Cauchy sequence has a limit; roughly, the space contains all necessary limits.
  • Polynomials on $[0,1]$ with the topology of uniform convergence are not complete: any continuous function on $[0,1]$ can be uniformly approximated by polynomials (Weierstrass approximation theorem).
  • The space of all continuous functions on $[0,1]$ with the same topology is complete.
• A norm gives a topology: $\mathbf{v}_n\to\mathbf{v}$ if and only if $\lim_{n\to\infty}|\mathbf{v}_n-\mathbf{v}|=0$.
• Banach and Hilbert spaces are complete topological vector spaces whose topologies come from a norm and an inner product respectively. Their study focuses on infinite-dimensional spaces, since all norms on a finite-dimensional topological vector space give the same notion of convergence (for instance the $1$-norm and $\infty$-norm on $\mathbf{R}^2$ have unit balls enclosing each other). In infinite dimensions, topologies are generally inequivalent, which makes the subject richer.
• Maps between topological vector spaces are required to be continuous. The topological dual space $V^*$ consists of continuous functionals $V\to\mathbf{R}$ (or $\mathbf{C}$). The Hahn-Banach theorem concerns separating subspaces by continuous functionals.

Banach spaces

Banach spaces, introduced by Stefan Banach, are complete normed vector spaces.

Example $\ell^p$: infinite vectors $\mathbf{x}=(x_1,x_2,\ldots,x_n,\ldots)$ with real entries and finite $p$-norm $(1\leq p\leq\infty)$:

$\|\mathbf{x}\|_p:=\left(\sum_i|x_i|^p\right)^{1/p}\ \ (p<\infty),\qquad \|\mathbf{x}\|_\infty:=\sup_i|x_i|.$

• The topologies on $\ell^p$ are inequivalent for different $p$.
• Example: $\mathbf{x}_n=(2^{-n},\ldots,2^{-n},0,0,\ldots)$ with the first $2^n$ components equal to $2^{-n}$. It converges to zero for $p=\infty$ but not for $p=1$:

$\|\mathbf{x}_n\|_\infty=2^{-n}\to0,\qquad \|\mathbf{x}_n\|_1=\sum_{i=1}^{2^n}2^{-n}=2^n\cdot2^{-n}=1.$

Lebesgue spaces $L^p(\Omega)$: for functions $f:\Omega\to\mathbb{R}$, replace the sum by the Lebesgue integral

$\|f\|_p:=\left(\int_\Omega|f(x)|^p\,d\mu(x)\right)^{1/p}.$

• $L^p(\Omega)$ consists of integrable functions with $\|f\|_p<\infty$ under this norm.
• These spaces are complete: if $\lim_{k,n\to\infty}\int_\Omega|f_k-f_n|^p\,d\mu=0$, there is $f\in L^p(\Omega)$ with $\lim_{k\to\infty}\int_\Omega|f-f_k|^p\,d\mu=0$. With the Riemann integral the space would not be complete, which can be seen as a justification for Lebesgue's theory.
• Imposing boundedness on derivatives as well leads to Sobolev spaces.

Hilbert spaces

Complete inner product spaces are Hilbert spaces, named for David Hilbert. A key case is $L^2(\Omega)$ with inner product

$\langle f,g\rangle=\int_\Omega f(x)\overline{g(x)}\,dx,$

where $\overline{g(x)}$ is the complex conjugate.

• Any Cauchy sequence converges to a limit.
• Approximation: Taylor approximation approximates differentiable functions by polynomials; by the Stone-Weierstrass theorem, every continuous function on $[a,b]$ can be approximated as closely as desired by a polynomial. Approximation by trigonometric functions is Fourier expansion, much applied in engineering.
• A set whose span has closure equal to the whole Hilbert space $H$ is a basis of $H$; its cardinality is the Hilbert space dimension. Together with the Gram-Schmidt process, this lets one construct a basis of orthogonal vectors, the generalization of coordinate axes in finite-dimensional Euclidean space.
• Physics link: solutions to many differential equations are interpreted in Hilbert spaces, often with orthogonal basis functions. The time-dependent Schrodinger equation describes physical change by a partial differential equation whose solutions are wavefunctions. Definite values of energy or momentum correspond to eigenvalues of a linear differential operator, with associated wavefunctions called eigenstates. The spectral theorem decomposes a linear compact operator in terms of eigenfunctions and eigenvalues.

Algebras over fields

General vector spaces have no multiplication of vectors. A vector space with an extra bilinear operator defining multiplication of two vectors is an algebra over a field ($F$-algebra).

• Polynomial ring: polynomials $p(t)$ form a vector space (sums are polynomials) and an algebra (products are polynomials). Rings of polynomials in several variables and their quotients are the basis of algebraic geometry.
• Lie algebras: neither commutative nor associative, but constrained by (with $[x,y]$ the product):
  • $[x,y]=-[y,x]$ (anticommutativity)
  • $[x,[y,z]]+[y,[z,x]]+[z,[x,y]]=0$ (Jacobi identity)
  • Examples: $n$-by-$n$ matrices with $[x,y]=xy-yx$ (the commutator), and $\mathbf{R}^3$ with the cross product.
• Tensor algebra $\operatorname{T}(V)$: a formal way to add products to any vector space. As a vector space it is spanned by simple tensors $\mathbf{v}_1\otimes\mathbf{v}_2\otimes\cdots\otimes\mathbf{v}_n$ of varying degree $n$. Multiplication is concatenation, with the distributive law and scalar multiplication commuting with $\otimes$. In general there are no relations between $\mathbf{v}_1\otimes\mathbf{v}_2$ and $\mathbf{v}_2\otimes\mathbf{v}_1$.
  • Forcing them equal gives the symmetric algebra.
  • Forcing $\mathbf{v}_1\otimes\mathbf{v}_2=-\mathbf{v}_2\otimes\mathbf{v}_1$ gives the exterior algebra.

---

🔗 Related Structures

Vector bundles

A vector bundle is a family of vector spaces parametrized continuously by a topological space $X$: a topological space $E$ with a continuous map $\pi:E\to X$ such that each fiber $\pi^{-1}(x)$ is a vector space.

• The case $\dim V=1$ is a line bundle.
• The projection $X\times V\to X$ gives the "trivial" vector bundle.
• Bundles are locally a product of $X$ and a fixed vector space $V$, but may be "twisted" globally. Example: the Mobius strip is a line bundle over the circle $S^1$, but differs from the cylinder $S^1\times\mathbf{R}$ because the cylinder is orientable and the strip is not.
• The tangent bundle collects the tangent spaces of a differentiable manifold. The tangent bundle of $S^1$ is globally isomorphic to $S^1\times\mathbf{R}$ (there is a global nonzero vector field), whereas by the hairy ball theorem the 2-sphere has no everywhere nonzero tangent vector field.
• K-theory studies isomorphism classes of vector bundles over a space; it has algebraic consequences such as the classification of finite-dimensional real division algebras: $\mathbf{R}$, $\mathbf{C}$, the quaternions $\mathbf{H}$ and the octonions $\mathbf{O}$.
• The cotangent bundle has, at each point, the dual of the tangent space (the cotangent space). Its sections are differential one-forms.

Modules

• Modules are to rings what vector spaces are to fields: the same axioms over a ring $R$ instead of a field $F$.
• Module theory is more complicated because some ring elements have no multiplicative inverses. Modules need not have bases (for example the $\mathbf{Z}$-module $\mathbf{Z}/2\mathbf{Z}$); those that do, including all vector spaces, are free modules.
• A vector space is compactly a module over a ring that is a field. Some authors use "vector space" for modules over a division ring.
• Locally free modules are the algebraic counterpart of vector bundles.

Affine and projective spaces

• Affine spaces are, roughly, vector spaces whose origin is not specified; precisely, a set with a free transitive vector space action. A vector space is an affine space over itself.
• An affine subspace of a vector space $W$ is a translate $x+V$ of a linear subspace $V$ by a fixed vector $x\in W$ (a coset of $V$ in $W$), consisting of all $x+v$ for $v\in V$.
• Important example: the solutions of an inhomogeneous system $A\mathbf{v}=\mathbf{b}$. The homogeneous case is $\mathbf{b}=\mathbf{0}$. The solution space is $x+V$, where $x$ is a particular solution and $V$ is the nullspace of $A$.
• Projective space: the set of one-dimensional subspaces of a fixed finite-dimensional vector space; it formalizes parallel lines intersecting at infinity. Grassmannians and flag manifolds generalize this by parametrizing subspaces of fixed dimension $k$ and flags of subspaces.