3.2 Basic operations

Transposition

It is often useful to switch the rows and columns of a matrix around. The resulting matrix is called the transpose of the original matrix, and denoted with a superscript \(^{\scriptstyle\top}\) or an apostrophe \('\):

\[ \mathbf{M}= \left[ \begin{array}{rr} 3 & 2 \\ 4 & -1 \\ -1 & 2 \end{array} \right] \qquad \mathbf{M}^{\scriptstyle\top}=\left[ \begin{array}{rrr} 3 & 4 & -1 \\ 2 & -1 & 2 \end{array} \right] \]

Note that \(M_{ij}=M ^{\scriptstyle\top}_{ji}\), and that if \(\mathbf{M}\) is an \(r \times c\) matrix, \(\mathbf{M}^{\scriptstyle\top}\) is a \(c \times r\) matrix.

Addition

There are two kinds of addition operations for matrices. The first is scalar addition:

\[ \mathbf{M}+ 2 = \left[ \begin{array}{rr} 3+2 & 2+2 \\ 4+2 & -1+2 \\ -1+2 & 2+2 \end{array} \right] = \left[ \begin{array}{rr } 5 & 4 \\ 6 & 1 \\ 1 & 4 \end{array} \right] \]

The other kind is matrix addition:

\[ \mathbf{M}+ \mathbf{M}= \left[ \begin{array}{rr} 3 & 2 \\ 4 & -1 \\ -1 & 2 \end{array} \right] + \left[ \begin{array}{rr} 3 & 2 \\ 4 & -1 \\ -1 & 2 \end{array} \right]= \left[ \begin{array}{rr} 6 & 4 \\ 8 & -2 \\ -2 & 4 \end{array} \right] \]

Formally, \((\mathbf{A}+\mathbf{B})_{ij} = A_{ij} + B_{ij}\).

Note that only matrices of the same dimension can be added to each other – there is no such thing as adding a \(4 \times 5\) matrix to a \(2 \times 9\) matrix.

Multiplication

There are also two common kinds of multiplication for matrices. The first is scalar multiplication:

\[ 4\mathbf{M}= 4\left[ \begin{array}{rr} 3 & 2 \\ 4 & -1 \\ -1 & 2 \end{array} \right] = \left[ \begin{array}{rr } 12 & 8 \\ 16 & -4 \\ -4 & 8 \end{array} \right] \]

Formally, \((c\mathbf{M})_{ij} = cM_{ij}\).

The other kind is matrix multiplication. The product of two matrices, \(\mathbf{A}\mathbf{B}\), is defined by multiplying all of \(\mathbf{A}\)’s rows by \(\mathbf{B}\)’s columns in the following manner:

\[ (\mathbf{A}\mathbf{B})_{ik} = \sum_j A_{ij}B_{jk} \]

\[ \left[ \begin{array}{rrr} 1 & 2 & 1 \\ 4 & -1 & 0 \end{array} \right] \left[ \begin{array}{rr} 3 & 2 \\ 0 & -1 \\ -1 & 2 \end{array} \right]= \left[ \begin{array}{rr} 2 & 2 \\ 12 & 9 \end{array} \right] \]

Note that matrix multiplication is only defined if the number of columns of \(\mathbf{A}\) matches the number of rows of \(\mathbf{B}\), and that if \(\mathbf{A}\) is an \(m \times n\) matrix and \(\mathbf{B}\) is an \(n \times p\) matrix, then \(\mathbf{A}\mathbf{B}\) is an \(m \times p\) matrix.

The following elementary algebra rules carry over to matrix algebra:

\[\begin{align*} \mathbf{A}+\mathbf{B}&= \mathbf{B}+\mathbf{A}& (\mathbf{A}+\mathbf{B})+\mathbf{C}&=\mathbf{A}+(\mathbf{B}+\mathbf{C}) \\ (\mathbf{A}\mathbf{B})\mathbf{C}&= \mathbf{A}(\mathbf{B}\mathbf{C}) & \mathbf{A}(\mathbf{B}+\mathbf{C})&=\mathbf{A}\mathbf{B}+\mathbf{A}\mathbf{C}\\ k(\mathbf{A}+\mathbf{B}) &= k\mathbf{A}+k\mathbf{B} \end{align*}\]

One important exception, however, is that \(\mathbf{A}\mathbf{B}\neq \mathbf{B}\mathbf{A}\); the order of matrix multiplication matters, and we must remember to, for instance, “left multiply” both sides of an equation by a matrix \(\mathbf{M}\) to preserve equality.

Inner and outer products

Suppose \(\mathbf{u}\) and \(\mathbf{v}\) are two \(n \times 1\) vectors. We can’t multiply them in the sense defined above, \(\mathbf{u}\mathbf{v}\), because the number of columns of \(\mathbf{u}\), 1, doesn’t match the number of rows of \(\mathbf{v}\), n. However, there are two ways in which vectors of the same dimension can be multiplied.

The first is called the inner product (also, the “cross product”):

\[\begin{align*} \mathbf{u}^{\scriptstyle\top}\mathbf{v}&= \sum_j u_j v_j \\ \left[\begin{array}{rr} 3 & 2 \end{array} \right] \left[\begin{array}{r} 2 \\ -1 \end{array}\right] &= 6 - 2 = 4. \end{align*}\]

Note that when we multiply matrices, the element \((\mathbf{A}\mathbf{B})_{ij}\) is equal to the inner product of the ith row of \(\mathbf{A}\) and the jth column of \(B\).

The second way of multiplying two vectors is called the outer product:

\[\begin{align*} (\mathbf{u}\mathbf{v}^{\scriptstyle\top})_{ij} &= u_i v_j \\ \left[\begin{array}{r} 3 \\ 2 \end{array} \right] \left[\begin{array}{rr} 2 & -1 \end{array}\right] &= \left[\begin{array}{rr} 6 & -3 \\ 4 & -2 \end{array}\right] \end{align*}\]

Note that the inner product returns a scalar number, while the outer product returns an \(n \times n\) matrix.