3.3 Special matrices

In the special case where a matrix has the same numbers of rows and columns, it is said to be square. If $\mathbf{A}^{\scriptstyle\top}=\mathbf{A}$, the matrix is said to be symmetric.

$$\text{Symmetric:} \left[ \begin{array}{rr} 1 & 2 \\ 2 & -1 \end{array} \right] \qquad \text{Not symmetric:} \left[ \begin{array}{rr} 3 & 2 \\ 0 & -1 \end{array} \right]$$

Note that a matrix cannot be symmetric unless it is square.

A matrix is said to be idempotent if $\mathbf{A}\mathbf{A}=\mathbf{A}$. Note that only square matrices can be idempotent. Idempotent matrices play a huge role in statistics, where they are typically referred to as projection matrices. Conceptually, the idea is that $\mathbf{A}$ defines a transformation and if $\mathbf{A}$ is idempotent, then applying it twice has the same effect as applying it once. In other words, you can project a vector onto a new space, but projecting it again doesn’t do anything, because it’s already been projected into that space.

The elements $A_{ii}$ of a matrix are called its diagonal entries; a matrix for which $A_{ij} = 0$ for all $i \neq j$ is said to be a diagonal matrix:

$$\left[ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 5 \end{array} \right]$$

Consider in particular the following diagonal matrix:

$$\mathbf{I}= \left[ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right]$$

Note that this matrix has the interesting property that $(\mathbf{A}\mathbf{I})_{ij}=A_{ij}$ for all $i, \, j$; in other words, $\mathbf{A}\mathbf{I}=\mathbf{I}\mathbf{A}=\mathbf{A}$. Because of this property, $\mathbf{I}$ is referred to as the identity matrix.

Some other notations which are commonly used are $\mathbf{1}$, the vector (or matrix) of 1s, and $\mathbf{0}$, the vector (or matrix) of zeros:

$$\mathbf{1}= \left[ \begin{array}{rrr} 1\ 1\ 1 \end{array} \right] \qquad \mathbf{0}= \left[ \begin{array}{rrr} 0\ 0\ 0 \end{array} \right]$$

The dimensions of these matrices is sometimes explicitly specified, as in $\mathbf{0}_{2 \times 2}$, $\mathbf{I}_{5 \times 5}$, or $\mathbf{1}_{4 \times 1}$. Other times it is obvious from context what the dimensions must be.

Finally, the vector $\mathbf{e}_j$ is also useful: it has element $e_j=1$ and $e_k=0$ for all other elements:

$$\mathbf{e}_2 = \left[ \begin{array}{rrr} 0 \\ 1 \\ 0 \end{array} \right].$$

This is useful for selecting a single element of a vector: $\mathbf{u}^{\scriptstyle\top}\mathbf{e}_3 = u_3$.