Matrices and Linear Systems

Matrices are the numerical engines of Linear Algebra. While a “Vector Space” is a theoretical playground, a Matrix is the specific blueprint that tells us how to manipulate that space.

1. The Matrix as a Map

A matrix $A$ of size $m \times n$ is a grid of numbers that represents a mapping from $\mathbb{R}^n$ to $\mathbb{R}^m$. Each column of the matrix tells us where one of the basis vectors of $\mathbb{R}^n$ “lands” in $\mathbb{R}^m$.

Let’s visualize a Shear Transformation matrix: $A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$. This matrix leaves the $x$-axis alone but shifts the $y$-direction.

import numpy as np
import matplotlib.pyplot as plt

# A 2x2 Shear Matrix
A = np.array([[1, 1], 
            [0, 1]])

# Create a grid of points (a square)
x, y = np.meshgrid(np.linspace(0, 1, 10), np.linspace(0, 1, 10))
points = np.vstack([x.flatten(), y.flatten()])

# Transform the points: A * points
transformed = A @ points

plt.figure(figsize=(8, 4))
plt.subplot(1, 2, 1)
plt.scatter(points[0], points[1], c='blue', s=5)
plt.title("Original Square")
plt.axis('equal')

plt.subplot(1, 2, 2)
plt.scatter(transformed[0], transformed[1], c='red', s=5)
plt.title("Transformed (Shear)")
plt.axis('equal')
plt.show()

2. Linear Systems: $Ax = b$

A system of linear equations asks: “Which vector $x$ lands on $b$ when we apply the transformation $A$?”

If the matrix $A$ squashes space into a lower dimension (i.e., it is Rank-Deficient or Singular), then $b$ might be unreachable, or there might be infinitely many paths to reach it.

import numpy as np

# A singular matrix (rows are multiples)
A = np.array([[1, 2], 
            [2, 4]])
b = np.array([5, 10])

try:
  x = np.linalg.solve(A, b)
  print(f"Solution: {x}")
except np.linalg.LinAlgError:
  print("Matrix is singular! No unique solution.")

# Check the rank
print(f"Rank of A: {np.linalg.matrix_rank(A)}")
print("Since Rank (1) < Dimensions (2), the matrix isn't 'full' and can't be inverted.")

3. Multiplication: Composition of Maps

Multiplying two matrices $AB$ represents applying transformation $B$ first, then $A$. Because the order of geometric transformations (like rotating then shifting) matters, matrix multiplication is not commutative ($AB \neq BA$).

4. Reduced Row Echelon Form (RREF)

RREF is the “simplest” version of a matrix that still represents the same linear system. It allows us to read off the solutions directly. In RREF, the leading entry of each row is 1, and all other entries in that column are 0.

5. Summary Check