LA Endterm — Pattern Recognition Cheatsheet

1. Write augmented matrix $[A \mid \mathbf{b}]$
2. Row reduce to RREF
3. No row $[0\ 0\ \dots\ 0 \mid d],\ d\neq 0$ means consistent
4. Free variables → parametric form

1. Multiply both sides by $A^{-1}$ on the correct side (left or right)
2. Key rules: $(AB)^{-1} = B^{-1}A^{-1}$, $(A^T)^{-1} = (A^{-1})^T$, $(AB)^T = B^TA^T$
3. Order matters — matrix multiplication is not commutative

1. Row reduce $A$ to echelon form
2. Identify pivot columns
3. Take those columns from the original $A$ (not the echelon form!)
4. Non-pivot column = linear combination of pivot columns (read from echelon form)

1. Eigenvalues: solve $\det(A - \lambda I) = 0$
2. Eigenvectors: solve $(A - \lambda I)\mathbf{x} = \mathbf{0}$, parametric form
3. Triangular matrix → eigenvalues are diagonal entries
4. Shortcut: if eigenvectors given, verify $A\mathbf{v} = \lambda\mathbf{v}$ to find $\lambda$

1. Find eigenvalues and their algebraic multiplicities
2. For each eigenvalue with a.m. > 1: find eigenspace dimension (g.m.)
3. Diagonalizable iff g.m. = a.m. for every eigenvalue
4. Shortcut: columns of $P$ must be eigenvectors — check $A\mathbf{v} = \lambda\mathbf{v}$

1. General solution: $\mathbf{x}_k = c_1\lambda_1^k\mathbf{v}_1 + c_2\lambda_2^k\mathbf{v}_2$
2. For IVP: plug $k=0$, set equal to $\mathbf{x}_0$, solve for $c_1, c_2$
3. Read off the component they ask for ($x_n$ or $y_n$)

1. Design matrix: $X = \begin{bmatrix} 1 & x_1 \\ 1 & x_2 \\ \vdots & \vdots \end{bmatrix}$, observation vector $\mathbf{y}$
2. Compute $X^TX$ and $X^T\mathbf{y}$
3. Solve $X^TX\boldsymbol{\beta} = X^T\mathbf{y}$
4. Answer: $y = \beta_0 + \beta_1 x$

1. $\mathbf{v}_1 = \mathbf{b}_1$
2. $\mathbf{v}_2 = \mathbf{b}_2 - \frac{\mathbf{b}_2 \cdot \mathbf{v}_1}{\mathbf{v}_1 \cdot \mathbf{v}_1}\mathbf{v}_1$
3. $\mathbf{v}_3 = \mathbf{b}_3 - \frac{\mathbf{b}_3 \cdot \mathbf{v}_1}{\mathbf{v}_1 \cdot \mathbf{v}_1}\mathbf{v}_1 - \frac{\mathbf{b}_3 \cdot \mathbf{v}_2}{\mathbf{v}_2 \cdot \mathbf{v}_2}\mathbf{v}_2$
4. If $\mathbf{v}_k = \mathbf{0}$ → that vector was dependent, skip it
5. Verify: $\mathbf{v}_i \cdot \mathbf{v}_j = 0$ for $i \neq j$

1. Put spanning vectors of $W$ as rows of a matrix
2. Find the null space
3. That null space $= W^\perp$

1. $r = |\lambda| = \sqrt{a^2+b^2}$, $\varphi = \arctan(b/a)$
2. $C = \begin{bmatrix} a & -b \\ b & a \end{bmatrix}$
3. Find eigenvector $\mathbf{v}$ for the complex eigenvalue: solve $(A-\lambda I)\mathbf{x}=\mathbf{0}$
4. $P = [\text{Re}(\mathbf{v}) \mid \text{Im}(\mathbf{v})]$

1. Take $\det$ of both sides
2. Use: $\det(AB) = \det A \cdot \det B$, $\det(A^T)=\det A$, $\det(A^{-1})=\tfrac{1}{\det A}$, $\det(cA)=c^n\det A$
3. Solve the resulting equation for $\det(A)$
4. Singular means $\det A = 0$

1. Symmetric → eigenspaces for different eigenvalues are orthogonal
2. If $E_\lambda$ is known, $E_\mu$ is its orthogonal complement
3. Put $E_\lambda$ basis vectors as rows, find null space

rank = number of pivot columns. dim Nul = $n$ − rank (number of columns minus pivots).

Cofactor expansion along row/column with most zeros. Or row reduce to triangular (track sign flips from swaps).

Key concepts you got confused on:

• $P$ is the machine, $\mathbf{y}$ is the input, $P\mathbf{y}$ is the shadow on $W$
• The $k$-th column of $P$ equals projecting $\mathbf{e}_k$ onto $W$
• You don't need to build the full matrix — just project the right $\mathbf{e}_k$

Steps:

1. Get an orthogonal basis for $W$ (Gram-Schmidt if needed)
2. Normalize to orthonormal: $\mathbf{u}_i = \mathbf{v}_i / \|\mathbf{v}_i\|$
3. $P = UU^T$ where $U = [\mathbf{u}_1 \mid \mathbf{u}_2 \mid \dots]$
4. Or use projection formula directly: $\displaystyle\text{proj}_W(\mathbf{y}) = \sum \frac{\mathbf{y}\cdot\mathbf{v}_i}{\mathbf{v}_i\cdot\mathbf{v}_i}\mathbf{v}_i$

What tripped you up:

• Row reducing with complex entries — don't bother, just use one row directly
• The system is guaranteed to have a free variable (eigenvalue means det = 0)
• Pick the cleaner row, express $x_1$ in terms of $x_2$, choose $x_2$ to cancel denominators

The trick you needed a hint for:

• Given $T$ on non-standard vectors → express $\mathbf{e}_1, \mathbf{e}_2$ as combinations of the given inputs
• Then use linearity: $T(\mathbf{e}_i) =$ same combination of the given outputs
• $A = [T(\mathbf{e}_1) \mid T(\mathbf{e}_2) \mid \dots]$
• NSI = the given vectors don't span the domain (can't express all standard basis vectors)

For geometric transformations (rotation/reflection):

• Rotation by $\theta$ ccw: $\begin{bmatrix}\cos\theta & -\sin\theta \\ \sin\theta & \cos\theta\end{bmatrix}$, clockwise: use $-\theta$
• Reflect x-axis: $\begin{bmatrix}1&0\\0&-1\end{bmatrix}$, y-axis: $\begin{bmatrix}-1&0\\0&1\end{bmatrix}$, y=x: $\begin{bmatrix}0&1\\1&0\end{bmatrix}$, y=−x: $\begin{bmatrix}0&-1\\-1&0\end{bmatrix}$
• Composition: multiply right-to-left (T₂ ∘ T₁ → A₂A₁)

Row reduce $[\mathbf{b}_1\ \mathbf{b}_2\ \dots \mid \mathbf{w}]$ to $[I \mid \mathbf{c}]$. Then $[\mathbf{w}]_\mathcal{B} = \mathbf{c}$.

• Common true: direct computation, use $P^2=P$, expand dot products, IMT
• Common false: try 2×2 matrices with simple entries (0s and 1s), upper triangular
• $(A+B)(A-B) \neq A^2 - B^2$ because $AB \neq BA$
• $AB = 0$ does NOT imply $BA = 0$