Matrix Multiplication: Linear Algebra Study Notes

🧮 Matrix Multiplication

Main Topics Covered

• Definition, size rule and notation of the matrix product
• Matrix times matrix, matrix times vector, vector times matrix, vector times vector
• Fundamental applications: linear maps, rotations, resource allocation, linear systems, bilinear forms
• General properties: non-commutativity, distributivity, scalar product, transpose, conjugate, associativity
• Computational cost of parenthesization and similarity transformations
• Square matrices: ring structure, inverses, determinant, powers
• Abstract algebra viewpoint
• Computational complexity (Strassen and beyond)
• Other types of matrix products

---

📌 Overview

Matrix multiplication is a binary operation that produces a matrix from two matrices. It is a basic tool of linear algebra.

• The number of columns in the first matrix must equal the number of rows in the second matrix.
• The resulting matrix, the matrix product, has the number of rows of the first and the number of columns of the second matrix.
• The product of matrices $\mathbf{A}$ and $\mathbf{B}$ is denoted $\mathbf{AB}$.
• It was first described by the French mathematician Jacques Philippe Marie Binet in 1812, to represent the composition of linear maps that are represented by matrices.
• Applications span mathematics, applied mathematics, statistics, physics, economics and engineering.
• Computing matrix products is a central operation in all computational applications of linear algebra.

🔤 Notation

• Matrices: capital letters in bold, e.g. $\mathbf{A}$.
• Vectors: lowercase bold, e.g. $\mathbf{a}$.
• Entries of vectors and matrices: italic (numbers from a field), e.g. $A$ and $a$.
• Index notation is often the clearest way to express definitions. The entry in row $i$, column $j$ of $\mathbf{A}$ is written $(\mathbf{A})_{ij}$, $A_{ij}$ or $a_{ij}$.
• A single subscript, e.g. $\mathbf{A}_1, \mathbf{A}_2$, selects a matrix (not an entry) from a collection of matrices.

📐 Definitions

Matrix times matrix

Let $\mathbf{A}$ be an $m \times n$ matrix and $\mathbf{B}$ an $n \times p$ matrix:

$\mathbf{A}=\begin{pmatrix}a_{11}&a_{12}&\cdots &a_{1n}\\a_{21}&a_{22}&\cdots &a_{2n}\\\vdots &\vdots &\ddots &\vdots \\a_{m1}&a_{m2}&\cdots &a_{mn}\end{pmatrix},\quad \mathbf{B}=\begin{pmatrix}b_{11}&b_{12}&\cdots &b_{1p}\\b_{21}&b_{22}&\cdots &b_{2p}\\\vdots &\vdots &\ddots &\vdots \\b_{n1}&b_{n2}&\cdots &b_{np}\end{pmatrix}$

The product $\mathbf{C}=\mathbf{AB}$ (written without multiplication signs or dots) is the $m \times p$ matrix

$\mathbf{C}=\begin{pmatrix}c_{11}&c_{12}&\cdots &c_{1p}\\c_{21}&c_{22}&\cdots &c_{2p}\\\vdots &\vdots &\ddots &\vdots \\c_{m1}&c_{m2}&\cdots &c_{mp}\end{pmatrix}$

such that

$c_{ij}=a_{i1}b_{1j}+a_{i2}b_{2j}+\cdots +a_{in}b_{nj}=\sum _{k=1}^{n}a_{ik}b_{kj}$

for $i = 1, \ldots, m$ and $j = 1, \ldots, p$.

• Each entry $c_{ij}$ is obtained by multiplying term by term the entries of the $i$th row of $\mathbf{A}$ and the $j$th column of $\mathbf{B}$, then summing these $n$ products.
• In other words, $c_{ij}$ is the dot product of the $i$th row of $\mathbf{A}$ and the $j$th column of $\mathbf{B}$.
• Written out in full:

$\mathbf{C}=\begin{pmatrix}a_{11}b_{11}+\cdots +a_{1n}b_{n1}&\cdots &a_{11}b_{1p}+\cdots +a_{1n}b_{np}\\\vdots &\ddots &\vdots \\a_{m1}b_{11}+\cdots +a_{mn}b_{n1}&\cdots &a_{m1}b_{1p}+\cdots +a_{mn}b_{np}\end{pmatrix}$

The product $\mathbf{AB}$ is defined if and only if the number of columns in $\mathbf{A}$ equals the number of rows in $\mathbf{B}$, in this case $n$.

Entries need not be numbers. They may be any mathematical objects for which an addition and a multiplication are defined, that are associative, and such that the addition is commutative and the multiplication is distributive with respect to the addition. In particular, the entries may be matrices themselves (block matrix).

Matrix times vector

• A vector $\mathbf{x}$ of size $n$ can be identified with an $n \times 1$ column matrix $\mathbf{X}$, with $\mathbf{X}_{i1}=\mathbf{x}_i$.
• For an $m \times n$ matrix $\mathbf{A}$, the product $\mathbf{Ax}$ is a vector $\mathbf{y}$ of size $m$ (the $m \times 1$ matrix $\mathbf{AX}$), with

$y_i=\sum _{j=1}^{n}a_{ij}x_j$

• One way of looking at this is that the changes from "plain" vector to column vector and back are assumed and left implicit.

Vector times matrix

• A vector $\mathbf{x}$ of size $n$ can also be identified with a $1 \times n$ (row) matrix.
• To make clear that a row vector is meant, it is customary to write it as the transpose of a column vector, as in $\mathbf{x}^{\mathrm{T}}\mathbf{A}$.
• Useful identity: $\mathbf{x}^{\mathrm{T}}\mathbf{A}=(\mathbf{A}^{\mathrm{T}}\mathbf{x})^{\mathrm{T}}$.
• For an $n \times p$ matrix $\mathbf{A}$, with $\mathbf{x}^{\mathrm{T}}\mathbf{A}=\mathbf{y}^{\mathrm{T}}$:

$y_k=\sum _{j=1}^{n}x_ja_{jk}$

Vector times vector

A vector with $n$ components can be represented as a $1 \times n$ matrix (row vector) or an $n \times 1$ matrix (column vector).

Product	Form	Result
Dot (inner) product of column vectors $\mathbf{a}, \mathbf{b}$	$\mathbf{a}^{\mathrm{T}}\mathbf{b}$	a $1 \times 1$ matrix
Outer product	$\mathbf{a}\mathbf{b}^{\mathrm{T}}$ (column times row)	an $n \times n$ matrix

Illustration

Each entry of the product matrix corresponds to a row of $\mathbf{A}$ and a column of $\mathbf{B}$. For a $4\times 2$ matrix times a $2\times 3$ matrix, giving a $4 \times 3$ matrix, the highlighted entries are:

$\begin{aligned}c_{12}&=a_{11}b_{12}+a_{12}b_{22}\\c_{33}&=a_{31}b_{13}+a_{32}b_{23}\end{aligned}$

🛠️ Fundamental Applications

Matrix multiplication was introduced to facilitate and clarify computations in linear algebra. This relationship remains fundamental in mathematics, physics, chemistry, engineering and computer science.

Linear maps

• If a vector space has a finite basis, each vector is uniquely represented by a finite sequence of scalars, its coordinate vector. These coordinate vectors form a vector space isomorphic to the original one.
• A coordinate vector is commonly organized as a column matrix, so a column vector represents both a coordinate vector and a vector of the original space.
• A linear map $A$ from a space of dimension $n$ into a space of dimension $m$ maps a column vector $\mathbf{x}$ onto the column vector

$\mathbf{y}=A(\mathbf{x})={\begin{pmatrix}a_{11}x_{1}+\cdots +a_{1n}x_{n}\\\vdots \\a_{m1}x_{1}+\cdots +a_{mn}x_{n}\end{pmatrix}}$

• The map is thus defined by the matrix $\mathbf{A}$ and maps $\mathbf{x}$ to $\mathbf{y}=\mathbf{Ax}$.
• If $B$ is another linear map into a space of dimension $p$, it is represented by a $p\times m$ matrix $\mathbf{B}$. The matrix of the composite map $B\circ A$ is the matrix product $\mathbf{BA}$.
• The definition of function composition, $(B\circ A)(\mathbf{x})=B(A(\mathbf{x}))$, is here a specific case of associativity of the matrix product:

$(\mathbf{BA})\mathbf{x}=\mathbf{B}(\mathbf{Ax})=\mathbf{BAx}$

Geometric rotations

In a Cartesian coordinate system in a Euclidean plane, rotation by an angle $\alpha$ around the origin is a linear map. It maps a source point $(x, y)$ to its image $(x', y')$ by

$\begin{bmatrix}x'\\y'\end{bmatrix}=\begin{bmatrix}\cos \alpha &-\sin \alpha \\\sin \alpha &\cos \alpha \end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix}$

The composition of the rotation by $\alpha$ and the rotation by $\beta$ is:

$\begin{bmatrix}\cos \beta &-\sin \beta \\\sin \beta &\cos \beta \end{bmatrix}\begin{bmatrix}\cos \alpha &-\sin \alpha \\\sin \alpha &\cos \alpha \end{bmatrix}=\begin{bmatrix}\cos(\alpha +\beta )&-\sin(\alpha +\beta )\\\sin(\alpha +\beta )&\cos(\alpha +\beta )\end{bmatrix}$

Trigonometric identities are used for the second equality. The composition corresponds to the rotation by angle $\alpha+\beta$, as expected.

Resource allocation in economics

A fictitious factory uses 4 kinds of basic commodities $b_1,\ldots,b_4$ to produce 3 kinds of intermediate goods $m_1, m_2, m_3$, which in turn are used to produce 3 kinds of final products $f_1, f_2, f_3$. The matrices

$\mathbf{A}=\begin{pmatrix}1&0&1\\2&1&1\\0&1&1\\1&1&2\end{pmatrix},\qquad \mathbf{B}=\begin{pmatrix}1&2&1\\2&3&1\\4&2&2\end{pmatrix}$

give the amount of basic commodities needed per intermediate good, and the amount of intermediate goods needed per final product, respectively.

• Example: to produce one unit of $m_1$, one unit of $b_1$, two units of $b_2$, no units of $b_3$ and one unit of $b_4$ are needed (the first column of $\mathbf{A}$).
• Multiplying gives the basic commodities needed per final good:

$\mathbf{AB}=\begin{pmatrix}5&4&3\\8&9&5\\6&5&3\\11&9&6\end{pmatrix}$

• The bottom left entry is $1\cdot 1+1\cdot 2+2\cdot 4=11$: 11 units of $b_4$ are needed to produce one unit of $f_1$. Indeed, one $b_4$ for $m_1$, one for each of two $m_2$, and 2 for each of four $m_3$ are needed for one $f_1$.
• To produce 100 units of $f_1$, 80 of $f_2$ and 60 of $f_3$:

$(\mathbf{AB})\begin{pmatrix}100\\80\\60\end{pmatrix}=\begin{pmatrix}1000\\1820\\1180\\2180\end{pmatrix}$

That is, 1000 of $b_1$, 1820 of $b_2$, 1180 of $b_3$ and 2180 of $b_4$. The same $\mathbf{AB}$ can be reused for other final-good amounts.

System of linear equations

The general form of a system of linear equations is

$\begin{matrix}a_{11}x_{1}+\cdots +a_{1n}x_{n}=b_{1},\\a_{21}x_{1}+\cdots +a_{2n}x_{n}=b_{2},\\\vdots \\a_{m1}x_{1}+\cdots +a_{mn}x_{n}=b_{m}\end{matrix}$

With the notation above, such a system is equivalent to the single matrix equation $\mathbf{Ax}=\mathbf{b}$.

Dot product, bilinear form and sesquilinear form

• The dot product of two column vectors is the unique entry of the matrix product $\mathbf{x}^{\mathsf{T}}\mathbf{y}$ (a $1\times1$ matrix is identified with its unique entry).
• Any bilinear form over a finite-dimensional vector space may be expressed as the matrix product $\mathbf{x}^{\mathsf{T}}\mathbf{Ay}$.
• Any sesquilinear form may be expressed as $\mathbf{x}^{\dagger}\mathbf{Ay}$, where $\mathbf{x}^{\dagger}$ is the conjugate transpose of $\mathbf{x}$ (conjugate of the transpose, or equivalently transpose of the conjugate).

⚖️ General Properties

Matrix multiplication shares some properties with usual multiplication. However, it is not defined if the number of columns of the first factor differs from the number of rows of the second, and it is non-commutative, even when the product remains defined after changing the order of the factors.

Non-commutativity

An operation is commutative if, given two elements $\mathbf{A}$ and $\mathbf{B}$ such that $\mathbf{AB}$ and $\mathbf{BA}$ are defined, $\mathbf{AB}=\mathbf{BA}$.

For $\mathbf{A}$ of size $m\times n$ and $\mathbf{B}$ of size $p\times q$:

• $\mathbf{AB}$ is defined iff $n=p$; $\mathbf{BA}$ is defined iff $m=q$. If one product is defined, the other need not be.
• If $m=q\neq n=p$, both are defined but have different sizes, so they cannot be equal.
• Only if $m=q=n=p$ (square matrices of the same size) are both products defined and of the same size. Even then, in general $\mathbf{AB}\neq\mathbf{BA}$.

Example:

$\begin{pmatrix}0&1\\0&0\end{pmatrix}\begin{pmatrix}0&0\\1&0\end{pmatrix}=\begin{pmatrix}1&0\\0&0\end{pmatrix}\quad\text{but}\quad\begin{pmatrix}0&0\\1&0\end{pmatrix}\begin{pmatrix}0&1\\0&0\end{pmatrix}=\begin{pmatrix}0&0\\0&1\end{pmatrix}$

• This can be expanded to show: for an $n\times n$ matrix $\mathbf{A}$ over a field $F$, $\mathbf{AB}=\mathbf{BA}$ for every $n\times n$ matrix $\mathbf{B}$ over $F$ if and only if $\mathbf{A}=c\,\mathbf{I}$ with $c\in F$, where $\mathbf{I}$ is the $n\times n$ identity matrix.
• Over a ring instead of a field, one must add the condition that $c$ belongs to the center of the ring.
• A special case where commutativity does occur: two square diagonal matrices $\mathbf{D}$ and $\mathbf{E}$ of the same size satisfy $\mathbf{DE}=\mathbf{ED}$ (over a general ring, corresponding entries must also commute).

Distributivity

The product is distributive with respect to matrix addition. For matrices of sizes $m\times n$, $n\times p$, $n\times p$ and $p\times q$ (call them $\mathbf{A},\mathbf{B},\mathbf{C},\mathbf{D}$):

• Left distributivity: $\mathbf{A}(\mathbf{B}+\mathbf{C})=\mathbf{AB}+\mathbf{AC}$
• Right distributivity: $(\mathbf{B}+\mathbf{C})\mathbf{D}=\mathbf{BD}+\mathbf{CD}$

These follow from distributivity for coefficients:

$\sum _{k}a_{ik}(b_{kj}+c_{kj})=\sum _{k}a_{ik}b_{kj}+\sum _{k}a_{ik}c_{kj}$

Product with a scalar

• For a matrix $\mathbf{A}$ and scalar $c$, $c\mathbf{A}$ and $\mathbf{A}c$ are obtained by left or right multiplying all entries of $\mathbf{A}$ by $c$. If scalars commute, $c\mathbf{A}=\mathbf{A}c$.
• If $\mathbf{AB}$ is defined, then $c(\mathbf{AB})=(c\mathbf{A})\mathbf{B}$ and $(\mathbf{AB})c=\mathbf{A}(\mathbf{B}c)$.
• If scalars commute, all four matrices are equal. More generally, all four are equal if $c$ belongs to the center of a ring containing the entries, because then $c\mathbf{X}=\mathbf{X}c$ for all matrices $\mathbf{X}$.
• These properties result from the bilinearity of the product of scalars:

$c\left(\sum _{k}a_{ik}b_{kj}\right)=\sum _{k}(ca_{ik})b_{kj}$

Transpose

If scalars commute, the transpose of a product is the product, in reverse order, of the transposes of the factors:

$(\mathbf{AB})^{\mathsf{T}}=\mathbf{B}^{\mathsf{T}}\mathbf{A}^{\mathsf{T}}$

where T denotes the interchange of rows and columns. This identity does not hold for noncommutative entries, since the order between the entries of $\mathbf{A}$ and $\mathbf{B}$ is reversed when the definition is expanded.

Complex conjugate

If $\mathbf{A}$ and $\mathbf{B}$ have complex entries, then

$(\mathbf{AB})^{*}=\mathbf{A}^{*}\mathbf{B}^{*}$

where $*$ denotes the entry-wise complex conjugate. This follows because the conjugate of a sum is the sum of the conjugates and the conjugate of a product is the product of the conjugates.

Transposition acts on the indices of the entries, while conjugation acts independently on the entries themselves. Hence

$(\mathbf{AB})^{\dagger}=\mathbf{B}^{\dagger}\mathbf{A}^{\dagger}$

where $\dagger$ denotes the conjugate transpose.

Associativity

Given three matrices, $(\mathbf{AB})\mathbf{C}$ and $\mathbf{A}(\mathbf{BC})$ are defined iff the columns of $\mathbf{A}$ equal the rows of $\mathbf{B}$ and the columns of $\mathbf{B}$ equal the rows of $\mathbf{C}$ (so if one product is defined, so is the other). Then:

$(\mathbf{AB})\mathbf{C}=\mathbf{A}(\mathbf{BC})$

• Parentheses can be omitted, writing $\mathbf{ABC}$.
• This extends to any number of matrices with matching dimensions: if the columns of $\mathbf{A}_i$ equal the rows of $\mathbf{A}_{i+1}$ for $i=1,\ldots,n-1$, then

$\prod _{i=1}^{n}\mathbf{A}_i=\mathbf{A}_1\mathbf{A}_2\cdots \mathbf{A}_n$

is defined and does not depend on the order of the multiplications, if the order of the matrices is kept fixed.

• These properties may be proved by straightforward but complicated summation manipulations. The result also follows from the fact that matrices represent linear maps, so associativity of matrices is a specific case of associativity of function composition.

Computational complexity depends on parenthesization

The result does not depend on the order of operation, but the computational complexity may depend dramatically on it.

Example: for $\mathbf{A},\mathbf{B},\mathbf{C}$ of sizes $10\times30$, $30\times5$, $5\times60$:

Order	Multiplications needed
$(\mathbf{AB})\mathbf{C}$	$10\times30\times5 + 10\times5\times60 = 4{,}500$
$\mathbf{A}(\mathbf{BC})$	$30\times5\times60 + 10\times30\times60 = 27{,}000$

Algorithms exist for choosing the best order of products (matrix chain multiplication). As the number $n$ of matrices increases, the choice of the best order has been shown to have a complexity of $O(n\log n)$.

Application to similarity

Any invertible matrix $\mathbf{P}$ defines a similarity transformation on square matrices of the same size:

$S_{\mathbf{P}}(\mathbf{A})=\mathbf{P}^{-1}\mathbf{A}\mathbf{P}$

Similarity transformations map products to products: $S_{\mathbf{P}}(\mathbf{AB})=S_{\mathbf{P}}(\mathbf{A})S_{\mathbf{P}}(\mathbf{B})$. Indeed,

$\mathbf{P}^{-1}(\mathbf{AB})\mathbf{P}=\mathbf{P}^{-1}\mathbf{A}(\mathbf{P}\mathbf{P}^{-1})\mathbf{B}\mathbf{P}=(\mathbf{P}^{-1}\mathbf{A}\mathbf{P})(\mathbf{P}^{-1}\mathbf{B}\mathbf{P})$

⬛ Square Matrices

Let $\mathcal{M}_n(R)$ denote the set of $n\times n$ square matrices with entries in a ring $R$ (in practice often a field).

• The product is defined for every pair of matrices in $\mathcal{M}_n(R)$, making it a ring with the identity matrix $\mathbf{I}$ (diagonal entries 1, all others 0) as identity element. It is also an associative $R$-algebra.
• If $n>1$, many matrices have no multiplicative inverse. For example, a matrix with a row (or column) of all zeros has no inverse.
• If it exists, the inverse of $\mathbf{A}$ is denoted $\mathbf{A}^{-1}$ and satisfies

$\mathbf{A}\mathbf{A}^{-1}=\mathbf{A}^{-1}\mathbf{A}=\mathbf{I}$

• A matrix with an inverse is invertible; otherwise it is singular.
• A product of matrices is invertible iff each factor is invertible, and then $(\mathbf{AB})^{-1}=\mathbf{B}^{-1}\mathbf{A}^{-1}$.
• When $R$ is commutative (in particular a field), the determinant of a product is the product of the determinants:

$\det(\mathbf{AB})=\det(\mathbf{BA})=\det(\mathbf{A})\det(\mathbf{B})$

• Other matrix invariants behave less well with products. Still, if $R$ is commutative, $\mathbf{AB}$ and $\mathbf{BA}$ have the same trace, the same characteristic polynomial, and the same eigenvalues with the same multiplicities. The eigenvectors are generally different if $\mathbf{AB}\neq\mathbf{BA}$.

Powers of a matrix

A square matrix can be raised to any nonnegative integer power by repeated multiplication:

$\mathbf{A}^0=\mathbf{I},\qquad \mathbf{A}^1=\mathbf{A},\qquad \mathbf{A}^k=\underbrace{\mathbf{A}\mathbf{A}\cdots \mathbf{A}}_{k\text{ times}}$

• The trivial algorithm (repeated multiplication) needs $k-1$ times the cost of a single matrix multiplication.
• Exponentiation by squaring requires fewer than $2\log_2 k$ matrix multiplications, and is much more efficient.
• Easy case, a diagonal matrix: the product of diagonal matrices just multiplies corresponding diagonal elements, so the $k$th power raises each entry to the power $k$:

$\begin{bmatrix}a_{11}&0&\cdots &0\\0&a_{22}&\cdots &0\\\vdots &\vdots &\ddots &\vdots \\0&0&\cdots &a_{nn}\end{bmatrix}^{k}=\begin{bmatrix}a_{11}^{k}&0&\cdots &0\\0&a_{22}^{k}&\cdots &0\\\vdots &\vdots &\ddots &\vdots \\0&0&\cdots &a_{nn}^{k}\end{bmatrix}$

🧩 Abstract Algebra

• The definition of the matrix product requires that entries belong to a semiring, and does not require multiplication of the semiring's elements to be commutative.
• Often entries belong to a field; the tropical semiring is also a common choice for graph shortest path problems.
• Even over fields, the product is not commutative in general, although it is associative and distributive over matrix addition.
• Identity matrices (zero outside the main diagonal, 1 on it) are identity elements of the matrix product. So $n\times n$ matrices over a ring form a ring, which is noncommutative except if $n=1$ and the ground ring is commutative.
• Over a commutative ring $R$, a matrix has an inverse iff its determinant has a multiplicative inverse in $R$. The determinant of a product of square matrices is the product of the determinants.
• The $n\times n$ invertible matrices form a group under matrix multiplication; its subgroups are called matrix groups. Many classical groups (including all finite groups) are isomorphic to matrix groups, the starting point of group representation theory.
• Matrices are the morphisms of a category, the category of matrices. Objects are the natural numbers measuring matrix size; composition of morphisms is matrix multiplication. The source of a morphism is the number of columns, and the target is the number of rows.

⏱️ Computational Complexity

• The algorithm that follows the definition needs, in the worst case, $n^3$ multiplications and $(n-1)n^2$ additions of scalars to multiply two $n\times n$ matrices. Its complexity is therefore $O(n^3)$, in a model where scalar operations take constant time.
• Surprisingly, this is not optimal: in 1969 Volker Strassen gave an algorithm (Strassen's algorithm) with complexity

$O(n^{\log _{2}7})\approx O(n^{2.8074})$

• Strassen's algorithm can be parallelized to further improve performance.
• As of January 2024, the best peer-reviewed algorithm, by Virginia Vassilevska Williams, Yinzhan Xu, Zixuan Xu and Renfei Zhou, has complexity $O(n^{2.371552})$.
• It is not known whether matrix multiplication can be done in $n^{2+o(1)}$ time. That would be optimal, since one must read the $n^2$ elements of a matrix to multiply it with another.
• Because matrix multiplication underlies many algorithms, and many matrix operations have the same complexity up to a multiplicative constant, its complexity appears throughout numerical linear algebra and theoretical computer science.

🔗 Generalizations

Other types of products of matrices:

Product	Description
Block matrix operations	Operations on matrices whose entries are matrices
Cracovian product	$\mathbf{A}\wedge\mathbf{B}=\mathbf{B}^{\mathsf{T}}\mathbf{A}$
Frobenius inner product	Dot product of matrices considered as vectors; equivalently the sum of the entries of the Hadamard product
Hadamard product	Entry-by-entry product of two matrices of the same size, giving a matrix of the same size
Kronecker (tensor) product	Generalization of the preceding to any size
Khatri-Rao product, face-splitting product	Further matrix products
Outer product (dyadic or tensor product)	Of two column matrices: $\mathbf{a}\mathbf{b}^{\mathsf{T}}$
Scalar multiplication	Multiplying every entry by a scalar