Linear Algebra Cheat Sheet (1)

0- Notations and convention

A variable/quantity/element can be scalar, vector, or a matrix; its type should be understood from the context if not explicitly declared.

0-1- A vector z\in \mathbb{F}^{n} is considered as a column vector, i.e. a single-column matrix. Therefore:

z\in \mathbb{F}^{n}\equiv z\in \mathbb{F}^{n\times 1}

A row vector is therefore defined as w^\text T\in \mathbb{F}^{1\times n}.

0-2- Dot product of column vectors:

\text{For }x,y\in \mathbb{R}^{n\times 1}, \ x\cdot y:=x^\text Ty

0-3- The ij-th element of a matrix A is denoted by A_{ij}. The i-th column and j-th row of the matrix are respectively denoted by A_{,i} and A_{j,} .

0-4- Columns and rows of a matrix are column and row vectors. A m\times n-matrix can then be represented as:

\begin{bmatrix} v_1& v_2& \dots & v_n\end{bmatrix}\ \text{ s.t } v_i=A_{,i}

or

\begin{bmatrix}u_1^\text T\\ u_2^\text T\\ \vdots\\ u_n^\text T\end{bmatrix}\ \text{ s.t } u_i^\text T=A_{i,}


1- Orthogonal matrix

a square matrix A\in \mathbb{R}^{n\times n} is said (definition) to be orthogonal iff A^{-1}=A^\text T; provided that A^{-1}, inverse of A exists. As a result, AA^{-1}=AA^\text T = I_n.

The following are equivalent for A:
a) A is orthogonal.
b) the column vectors of A are ortho-normal.
c) the row vectors of A are ortho-normal.
d) A is size preserving: \|Ax\|=\|x\|, \|.\| being the Euclidean norm, and x\in\ \mathbb{R}^{n\times 1}.
e) A is dot product preserving: Ax\cdot Ay=x\cdot y .


2- Some matrix multiplication identities

A\in \mathbb{F}^{m\times n},\ x\in \mathbb{F}^{n\times 1}\ \text{ then } Ax=\sum_{i=1}^n x_iA_{,i} A\in \mathbb{F}^{m\times n}\ , D\in \mathbb{F}^{n\times n}\ \text{ and diagonal }, B\in \mathbb{F}^{n\times k},\ \text{ s.t.}\\ \ \\ A=\begin{bmatrix} v_1& v_2& \dots & v_n\end{bmatrix}, \ B=\begin{bmatrix}u_1^\text T\\ \\ u_2^\text T\\ \vdots\\ u_n^\text T\end{bmatrix},\ D_{ij}= \begin{cases} \lambda_i &\text{, } i=j \\ 0 &\text{, } i\ne j \end{cases} \\ \ \\ \text{then }ADB=\sum_{i=1}^n \lambda_iv_i u_i^\text T

3- Change of basis

let \beta:=\{b_1,\dots,b_n\} and \beta’:=\{b’_1,\dots,b’_n\} be two basis of \mathbb R^n. Then the following holds for the coordinates of a vector v\in\mathbb R^n with respect to the two bases:

[v]_{\beta’}=P_{\beta\to\beta’}[v]_{\beta}\quad\text{s.t} \\ \ \\ P_{\beta\to\beta’}=\begin{bmatrix} [b_1]_{\beta’}& [b_2]_{\beta’}& \dots & [b_n]_{\beta’}\end{bmatrix}

It can be proved that:

[v]_{\beta}=P_{\beta’\to\beta}[v]_{\beta’}\quad\text{s.t} \\ \ \\ P_{\beta’\to\beta}=P^{-1}_{\beta\to\beta’}