Review of Matrix Algebra

If n and m are positive integers, an $m\times n$ matrix is a function from the set of integer ordered pairs (i,j) with $1 \leq i \leq m$ and $1 \leq j \leq n$ to a given set. If this set is a subset of the complex numbers, the matrix is called a matrix over the complex numbers or a complex matrix. If this set is a subset of the real numbers, the matrix is called a matrix over the real numbers or a real matrix. We can have polynomial matrices, etc. If the matrix is called A, then A(i,j) is called the (i,j) entry of A. An $m\times n$ array is typically pictured as an array with m rows and n columns, and A(i,j) is interpreted as the entry in the i'th row and j'th column.

A $m\times n$ matrix is called em square if m = n. The transpose of the $m\times n$ matrix A, denoted by A^t, is the $n\times m$ matrix with A^t(i,j) = A(j,i), i = 1,...,n and j = 1,...,m. A real valued matrix A is called symmetric if A = A^t. A square matrix A is called diagonal if $i\neq j$ implies A(i,j) = 0. A square matrix is called upper triangular if i < j implies A(i,j) = 0, and is called lower triangular if its transpose is upper triangular.

If A and B are both $m\times n$ matrices and for each pair (i,j) the quantity A(i,j) + B(i,j) makes sense (for example if the matrices are polynomial matrices) we can define the sum of the matrices, A+B, by (A+B)(i,j) = A(i,j) + B(i,j). This is just the usual definition of the sum of two functions with a common domain. Similarly, if t is any quantity for which tA(i,j) makes sense for all ordered pairs (i,j) we may define a matrix tA by (tA)(i,j) = tA(i,j). This is just the usual definition of the product of a function by a scalar.

In what follows we will assume that our matrices are complex matrices, and all multipliers t are complex numbers. We may occasionally restrict to real matrices and real multipliers.

If A is an $m\times n$ matrix and B is an $n\times p$ matrix we will define the matrix product AB to be the matrix with

$$(AB)(i,j) = \sum_{a=1}^n A(i,a)B(a,j)$$

It is tedious to show that if A, B and C are matrices with appropriate dimensions and t is a complex number then

• A(B+C) = AB + AC;
• (B+C)A = BA + CA;
• A(BC) = (AB)C;
• (A+B) + C = A + (B+C);
• t(AB) = (tA)B = A(tB);
• t(A+B) = (tA) + (tB);

It is easy enough to construct examples of $d\times d$ matrices A and B where $AB \neq BA$.

Let I be a $d\times d$ matrix with I(i,i) = 1 and I(i,j) = 0 for $i\neq j$.I is called the identity matrix. It has the property that if A is $d\times p$then IA = A, and if B is $p\times d$ then BI = B.

A matrix A with A(i,j) = 0 for all i and j is called a zero matrix.

A $d\times 1$ matrix is called a column vector and a $1\times d$ matrix is called a row vector.

If A is a square matrix, t is a complex number, and X is a non-zero column matrix such that AX = tX, then X is called a left eigenvector for A with eigenvalue t. If Y is a non-zero row vector such that YA = tY, then Y is called a right eigenvector for A with eigen value t. If X and Y are real column matrices of the same dimension then X and Y are said to be orthogonal if X^tY = 0. X is said to be a unit vector if X^tX = 1. A set of column vectors is said to be orthonormal if each vector in the set is a unit vector and any pair of distinct vectors is orthogonal.

To each $d\times d$ matrix A we may associate d column vectors $A^{(1)},\dots,A^{(d)}$ by the rule A^(j)(i,1) = A(i,j). These column vectors are called the columns of A. The row vectors and rows are defined in an analogous manner. A real valued square matrix $\cal O$ is called orthogonal if its columns form an orthonormal set. For any orthogonal matrix $\cal O$ we have ${\cal O}^t{\cal O} = I$.

The determinant function of order d, $\det_d$, is the unique complex valued function acting on the set of all $d\times d$ matrices with the properties

• As a function of any one column, $\det_d$ is linear.
• If the matrix B is obtained from A by interchanging one pair of columns, then $\det_d(B) = -det_d(A)$
• det_d(I) = 1.

Determinants are computed either by recursion by a procedure known as cofactor expansions, or by using the properties listed above. Note that the second property implies that a matrix with two equal columns must have a determinant of 0. Two important properties of determinants are that if A and B are $d\times d$ matrices then $\det_d(AB) = \det_d(A)\det_d(B)$ and that A and the transpose of A have the same determinant.

If A is a square matrix then A is said to be invertible if there is another square matrix B with AB = BA = I. A can have a most one inverse, and it is denoted by A^-1. Three important facts about a square matrix A are

• A has an inverse if and only the determinant of A is not 0;
• AX = 0 implies X = 0 for all X if and only if A has an inverse;
• t is an eigenvalue of A if and only if $\det_d(tI-A) = 0$.

Every orthogonal matrix is invertible, and its transpose is its inverse. The determinant of an orthogonal matrix is either 1 or -1.

If A is a symmetric matrix then there is an orthogonal matrix $\cal O$ whose columns are right eigenvectors of A. In this case the matrix ${\cal O}^tA{\cal O}$ is a diagonal matrix whose with ${\cal O}^tA{\cal O}(i,i)$ equal to the eigenvalue corresponding to the i'th column of $\cal O$.A symmetric matrix is called positive definite if all of its eigenvalues are positive. All of the eigenvalues of a symmetric matrix are real numbers.